Stereoscopic image pickup unit, image pickup device, picture processing method, control method, and program utilizing diaphragm to form pair of apertures

ABSTRACT

An image pickup unit, an image pickup device, a picture processing method, a diaphragm control method, and program are capable of suppressing deterioration in quality of a stereoscopic picture. A parallax detection pixel receives object light by a plurality of photodetectors covered with one microlens, to generate a signal used for detecting parallax. G pixels, an R pixel, and a B pixel each receive the object light to generate a signal used for generating a planar picture. A parallax detection section detects parallax based on the signal generated by the parallax detection pixels. A 2D picture generation section generates a planar picture based on a signal generated by picture generation pixels. A 3D picture generation section adjusts a position of each object image included in the planar picture, based on the detected parallax, to generate a stereoscopic picture.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of application Ser. No.14/002,028, filed Aug. 28, 2013, which is a National Stage ofPCT/JP2012-057332, filed Mar. 22, 2013, which claims the benefit ofJapanese Priority Patent Application(s) JP 2011-071605 and JP2011-071606, both filed Mar. 29, 2011, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an image pickup unit, and inparticular, to an image pickup unit that generates a stereoscopicpicture, an image pickup device, a picture processing method, and adiaphragm control method, and a program that causes a computer toexecute the methods.

BACKGROUND ART

In the past, there have been proposed image pickup units such as adigital still camera and a digital video camera (camera-integratedrecorder) that generate picture data used for stereoscopic picturedisplay that provides stereoscopic viewing with use of parallax betweenright and left eyes.

For example, an image pickup unit that includes two lenses and one imagepickup device and generates two pictures (a left-eye picture and aright-eye picture) for stereoscopic picture display has been proposed(for example, see PTL 1).

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2004-309868

SUMMARY OF INVENTION Technical Problem

According to the above-described existing technology, the two pictures(the left-eye picture and the right-eye picture) are generated with useof the two lenses and the one image pickup device. However, the imagepickup unit uses a polarization filter, and therefore light amount maybe decreased. In addition, it is possibly difficult to receive objectlight containing specific polarization (for example, reflected lightfrom glass and reflected light from a water surface).

Moreover, the two lenses provided cause a complicate optical system.Therefore, an image pickup unit that generates a stereoscopic picturewith use of one lens has also been proposed. In such an image pickupunit, however, object light is divided into left light and right lightby the one lens, and thus if adjustment of brightness is prioritizedwhile a diaphragm is stopped down, stereoscopic effect is diminished. Inother words, to improve the quality of a stereoscopic picture, it isnecessary to adjust brightness without diminishing stereoscopic effect.

The technology is made in view of the above circumstances, and it is anobject of the technology to suppress deterioration in quality of astereoscopic picture, and to improve picture quality.

Solution to Problem

The technology is provided to dissolve the above-described issue, and afirst aspect thereof is an image pickup unit, a picture processingmethod, and a program causing a computer to execute the method. Theimage pickup unit includes: an image pickup device including parallaxdetection pixels and picture generation pixels, each of the parallaxdetection pixels receiving object light by a plurality of photodetectorscovered by one microlens, to generate a signal used for detectingparallax, and each of the picture generation pixels receiving the objectlight to generate a signal used for generating a planar picture; and astereoscopic picture generation section detecting the parallax based onthe signal generated by the parallax detection pixels, generating theplanar picture based on the signal generated by the picture generationpixels, and adjusting a position of each object image included in thegenerated planar picture, based on the detected parallax, to generate astereoscopic picture. With this configuration, there is provided afunction of: detecting parallax based on a signal generated by parallaxdetection pixels; generating a planar picture based on a signalgenerated by picture generation pixels; and adjusting a position of eachobject image included in the planar picture based on the detectedparallax, to generate a stereoscopic picture.

In addition, in the first aspect, a posture detection section detectinga posture of the image pickup unit may be further included, the parallaxdetection pixels may be arranged on a line in a row direction of theimage pickup device and on a line in a column direction thereof, thestereoscopic picture generation section may determine a detectiondirection of the parallax from the row direction and the columndirection of the image pickup device, based on the posture detected bythe posture detection section, and may generate information related tothe parallax, based on the signal generated by the parallax detectionpixels arranged in the determined direction. With this configuration,there is provided a function of determining a detection direction ofparallax based on a posture detected by a posture detection section, andgenerating information related to the parallax, based on a signalgenerated by parallax detection pixels arranged in the determineddirection.

Moreover, in the first aspect, a focus determination section thatperforms focus determination on an object to be focused, based on thesignal generated by the parallax detection pixels may be furtherincluded. With this configuration, there is provided a function ofperforming focus determination on an object to be focused, based on asignal generated by parallax detection pixels.

Moreover, in the first aspect, the parallax detection pixels in theimage pickup device may be arranged to be adjacent to one another on aline in a specific direction. With this configuration, there is provideda function that parallax detection pixels are arranged to be adjacent toone another on a line in a specific direction.

Moreover, in the first aspect, the parallax detection pixels in theimage pickup device may be arranged with predetermined intervals on aline in a specific direction. With this configuration, there is provideda function that parallax detection pixels are arranged in an islandshape.

Moreover, in the first aspect, a control section that moves the onemicrolens covering the plurality of photodetectors in the parallaxdetection pixel, in an optical axis direction of the microlens, based ona relationship between the image pickup device and a size of the exitpupil may be further provided. With this configuration, there isprovided a function of detecting parallax with respect to a plurality ofexit pupils with different sizes.

Moreover, in the first aspect, the plurality of photodetectors in theparallax detection pixel may be covered with one color filter. As aresult, there is provided a function of allowing a plurality ofphotodetectors in a parallax detection pixel to have the same spectralcharacteristics. In addition, in this case, the plurality ofphotodetectors in the parallax detection pixel may be covered with agreen filter that shields light other than light within a wavelengthrange showing green. With this configuration, there is provided afunction of providing a green filter as a color filter for a pluralityof photodetectors in a parallax detection pixel.

Furthermore, in this case, the plurality of photodetectors in theparallax detection pixel may be covered with a white filter or atransparent layer, the white filter and the transparent layer allowinglight within a visible light range to pass therethrough. With thisconfiguration, there is provided a function of providing a white filteror a transparent layer as a color filter for a plurality ofphotodetectors in a parallax detection pixel.

Moreover, in the first aspect, the picture generation pixel may includeone photodetector on a pixel basis. With this configuration, there isprovided a function of generating a planar picture based on a signalgenerated by a picture generation pixel including one photodetector on apixel basis.

Moreover, in the first aspect, a microlens that is for collecting theobject light on a position of each of the plurality of photodetectorsmay cover the plurality of photodetectors on the plurality ofphotodetectors basis, the object light being collected by the onemicrolens covering the plurality of photodetectors in the parallaxdetection pixel. With this configuration, there is provided a functionthat a parallax detection pixel is provided with one microlens coveringa plurality of photodetectors and a microlens for collecting objectlight collected by the one microlens, on a position of each of theplurality of photodetectors.

Moreover, in the first aspect, the microlenses each covering thephotodetector in the picture generation pixel may be arranged on onesurface orthogonal to an optical axis direction of the microlenscovering the plurality of photodetectors in the parallax detection pixelon the plurality of photodetectors basis. With this configuration, thereis provided a function that a microlens covering each of a plurality ofphotodetectors in a parallax detection pixel and a microlens in apicture generation pixel are provided in the same layer.

Moreover, in the first aspect, the microlenses each covering thephotodetector in the picture generation pixel may be arranged on onesurface orthogonal to an optical axis of the one microlens covering theplurality of photodetectors in the parallax detection pixel. With thisconfiguration, there is provided a function that one microlens coveringa plurality of photodetectors in a parallax detection pixel and amicrolens of a picture generation pixel are provided in the same layer.

Moreover, according to a second aspect of the technology, there isprovided an image pickup device including parallax detection pixels eachreceiving object light by a plurality of photodetectors covered by onemicrolens, to generate a signal used for detecting parallax, theparallax being used for generating a stereoscopic picture; and picturegeneration pixels each receiving the object light by a photodetectorcovered, on a pixel basis, with a microlens smaller in size than themicrolens, to generate a signal used for generating a planar picture,the planar picture being used for generating the stereoscopic pictureusing the parallax. As a result, there is provided a function ofallowing an image pickup device to include parallax detection pixelseach including a plurality of photodetectors covered with one microlensand picture generation pixels each including a photodetector covered bya small microlens on a pixel basis.

Moreover, in the second aspect, the parallax may be information relatedto a displacement amount of a position of each object image in theplanar picture at a time when the position of each object image isadjusted in the parallax direction to generate the stereoscopic picture,and the parallax detection pixels may be arranged on a line in theparallax direction. With this configuration, there is provided afunction that a signal from parallax detection pixels arranged on a linein a parallax direction is used to calculate displacement amount of aposition of each object image in a planar picture.

Furthermore, according to a third aspect of the technology, there areprovided an image pickup unit, a diaphragm control method related to theimage pickup unit, and program causing a computer to execute the method.The image pickup unit includes: a diaphragm forming a pair of aperturesfor generating a stereoscopic picture; an image pickup device receivingobject light that passes through each of the pair of apertures, togenerate signals used for generating the stereoscopic picture; and acontrol section independently controlling a distance between centroidsof the pair of apertures, and increase and decrease of light amount ofthe object light passing through the pair of apertures. With thisconfiguration, there is provided a function of receiving object lightthat passes through a diaphragm forming a pair of apertures forgenerating a stereoscopic picture and generating a stereoscopic picture.

Moreover, in the third aspect, the pair of apertures may be formed, inthe diaphragm, to be adjacent to each other in a parallax direction ofthe stereoscopic picture, and the control section may change andcontrol, of peripheral edges of the pair of apertures, positions of endparts of the peripheral edges corresponding to both ends in the parallaxdirection and positions of closed parts of the peripheral edges that areclose to each other between the pair of apertures. With thisconfiguration, there is provided a function of: forming a pair ofapertures adjacent to each other in a parallax direction of astereoscopic picture; and changing and controlling positions of endparts of peripheral edges corresponding to both ends in the parallaxdirection and positions of closed parts of the peripheral edges that areclose to each other between the pair of apertures.

Moreover, in the third aspect, when the light amount is increased ordecreased, the control section may vary a length between the end part ofthe peripheral edge corresponding to the end of one of the pair ofapertures and the closed part of the peripheral edge thereof, and alength between the end part of the peripheral edge corresponding to theend of the other of the pair of apertures and the closed part of theperipheral edge thereof, in a state where the distance between thecentroids is fixed. With this configuration, there is provided afunction of, when a light amount is increased or decreased, varying alength between an end part of a peripheral edge corresponding to an endof one of a pair of apertures and a closed part of the peripheral edgethereof, and a length between an end part of a peripheral edgecorresponding to an end of the other of the pair of apertures and aclosed part of the peripheral edge thereof, in a state where a distancebetween centroids is fixed.

Moreover, in the third aspect, the length between the end part of theperipheral edge corresponding to the end of the one of the pair ofapertures and the closed part of the peripheral edge thereof may beequal to the length between the end part of the peripheral edgecorresponding to the end of the other of the pair of apertures and theclosed part of the peripheral edge thereof. With this configuration,there is provided a function of allowing length between an end part of aperipheral edge corresponding to an end of one of a pair of aperturesand a closed part of the peripheral edge thereof to be equal to a lengthbetween an end part of a peripheral edge corresponding to an end of theother of the pair of apertures and a closed part of the peripheral edgethereof.

Moreover, in the third aspect, when the distance between the centroidsis varied, the control section may vary the distance between thecentroids in a state where the length between the end part of theperipheral edge corresponding to the end of the one of the pair ofapertures and the closed part of the peripheral edge thereof is fixed.With this configuration, there is provided a function of, when adistance between centroids is varied, varying the distance between thecentroids in a state where a length between an end part of a peripheraledge corresponding to an end of one of a pair of apertures and a closedpart of the peripheral edge thereof is fixed.

Moreover, in the third aspect, an adjusting section adjusting thedistance between the centroids may be further provided, and the controlsection may control the pair of apertures to allow the distance betweenthe centroids to be a distance adjusted by the adjusting section. Withthis configuration, there is provided a function that a pair ofapertures is controlled so as to allow a distance between centroids tobe a distance adjusted by an adjusting section that adjusts the distancebetween the centroids.

Moreover, in the third aspect, the diaphragm may include a first memberthat includes a pair of members each having a cutout and a second memberthat includes a pair of members each having a projection, the pair ofmembers of the first member being disposed to allow the cutouts to faceeach other, and the pair of members of the second member being disposedto allow the projections to face each other. With this configuration,there is provided a function that a diaphragm is configured of a firstmember that includes a pair of members each having a cutout and a secondmember that includes a pair of members each having a projection, thepair of members of the first member being disposed to allow the cutoutsto face each other, and the pair of members of the second member beingdisposed to allow the projections to face each other.

Moreover, in the third aspect, the first member and the second membermay be driven in an orthogonal direction orthogonal to the parallaxdirection. With this configuration, there is provided a function ofdriving a first member and a second member in an orthogonal directionorthogonal to a parallax direction.

Moreover, in the third aspect, the cutout may have a concave shape inwhich an apex of a mountain shape is located on a straight line thatpasses through a midpoint of the distance between the centroids and isparallel to a driving direction of the first member, and the projectionmay have a convex shape in which an apex of a mountain shape is locatedon a straight line that passes through the midpoint of the distancebetween the centroids and is parallel to a driving direction of thesecond member. With this configuration, there is provided a function ofdriving a first member having a cutout of a concave shape in which anapex of a mountain shape is located on a straight line that passesthrough a midpoint of a distance between centroids and is parallel to adriving direction of the first member and a second member having aprojection of a convex shape in which an apex of a mountain shape islocated on a straight line that passes through the midpoint of thedistance between the centroids and is parallel to a driving direction ofthe second member.

Moreover, in the third aspect, a posture detection section detecting aposture of the image pickup unit may be further included. In addition,in the third aspect, the diaphragm may include a first member, a secondmember shielding part of the object light in a horizontal shooting, anda third member shielding part of the object light in a verticalshooting, the first member having a pair of members each having acutout, the pair of members of the first member being disposed to allowthe cutouts to face each other, the second member having a pair ofmembers each having a projection, the pair of members of the secondmember being disposed to allow the projections to face each other, thethird member having a pair of members each having a projection, and thepair of the members of the third member being disposed to allow theprojections to face each other. In addition, in the third aspect, adriving direction of the second member may be orthogonal to a drivingdirection of the third member. In addition, in the third aspect, thecontrol section may determine, based on the detected posture, whetherthe horizontal shooting or the vertical shooting is performed, and thencontrols the pair of apertures. With this configuration, there isprovided a function of forming a pair of apertures in a parallaxdirection both in horizontal shooting and in vertical shooting.

Moreover, in the third aspect, the diaphragm may be disposed on anoptical path of the object light that is collected by a monocular lenssystem. With this configuration, there is provided a function ofallowing a diaphragm to be disposed on an optical path of object lightthat is collected by a monocular lens system.

Furthermore, according to a fourth aspect of the technology, there isprovided an image pickup unit including: a diaphragm configured of apair of members each having a pair of cutouts that are adjacent to eachother in a parallax direction of a stereoscopic picture, the cutouts ofthe respective members facing to each other to form a pair of apertures;an image pickup device receiving object light that passes through eachof the pair of apertures to generate a signal used for generating thestereoscopic picture; and a control section driving each of the pair ofmembers in an orthogonal direction orthogonal to the parallax directionand controlling the diaphragm to allow a distance between centroids ofthe pair of apertures to be fixed. With this configuration, there isprovided a function of receiving object light that passes through a pairof apertures of a diaphragm that is configured of a pair of members eachincluding a pair of cutouts adjacent to each other in a parallaxdirection of a stereoscopic picture to generate a stereoscopic picture.

Furthermore, according to a fifth aspect of the technology, there isprovided an image pickup unit including: a diaphragm forming anaperture, a longitudinal direction of the aperture being a parallaxdirection in a stereoscopic picture; an image pickup device receivingobject light that passes through the aperture, to generate a signal usedfor generating the stereoscopic picture; and a control sectioncontrolling the diaphragm to allow a length of the aperture in theparallax direction to be larger than a length of the aperture in anorthogonal direction orthogonal to the parallax direction. With thisconfiguration, there is provided a function of receiving object lightthat passes through an aperture of a diaphragm that forms the aperturewhose longitudinal direction is a parallax direction of a stereoscopicpicture and accordingly generating the stereoscopic picture.

Moreover, in the fifth aspect, the diaphragm may include a pair ofmembers each having a cutout, the cutouts facing to each other to formthe aperture, and the control section may drive each of the pair ofmembers in the orthogonal direction to control the diaphragm. With thisconfiguration, there is provided a function of receiving object lightthat passes through an aperture formed by facing the pair of memberseach having a cutout and accordingly generating a stereoscopic picture.

Moreover, in the fifth aspect, the cutout may have one of a rectangularshape having a long side extending in the parallax direction, atriangular shape having a base extending in the parallax direction, anda semicircular shape having one side extending in the parallaxdirection. With this configuration, there is provided a function ofreceiving object light, which passes through an aperture formed of acutout having one of a rectangular shape having a long side extending ina parallax direction, a triangular shape having a base extending in theparallax direction, and a semicircular shape having one side extendingin the parallax direction, and accordingly generating a stereoscopicpicture.

Moreover, in the fifth aspect, the diaphragm may form the aperture withuse of a pair of first members and a pair of second members, the firstmembers each having a side parallel to the parallax direction, the sidesof the respective first members facing to each other, the second memberseach having a side parallel to the orthogonal direction, and the sidesof the respective second members facing to each other. With thisconfiguration, there is provided a function of receiving object light,which passes through an aperture formed by a pair of first members and apair of second members, and accordingly generating a stereoscopicpicture, the first members each having a side parallel to the parallaxdirection, the sides of the respective first members facing to eachother, the second members each having a side parallel to the orthogonaldirection, and the sides of the respective second members facing to eachother.

Advantageous Effects of Invention

According to the technology, it is possible to provide effects ofsuppressing deterioration in picture quality of a stereoscopic picture,and improving the picture quality.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 100 according to a firstembodiment of the technology.

FIG. 2 is a schematic diagram illustrating an example of an arrangementof pixels provided in an image pickup device 200 according to the firstembodiment of the technology.

FIG. 3 is a schematic diagram illustrating an example of a picturegeneration pixel and a parallax detection pixel that are provided in theimage pickup device 200 according to the first embodiment of thetechnology.

FIG. 4 is a schematic diagram illustrating an example of across-sectional structure of the picture generation pixel and theparallax detection pixel according to the first embodiment of thetechnology.

FIG. 5 is a diagram schematically illustrating object light received bythe parallax detection pixel according to the first embodiment of thetechnology.

FIG. 6 is a diagram schematically illustrating a principle of parallaxdetection by a parallax detection pixel 230 according to the firstembodiment of the technology.

FIG. 7 is a diagram schematically illustrating an example of a directionof the parallax detection by the parallax detection pixel 230 in thecase where horizontal shooting is performed using the image pickup unit100 according to the first embodiment of the technology.

FIG. 8 is a diagram schematically illustrating an example of thedirection of the parallax detection by the parallax detection pixel 230in the case where vertical shooting is performed using the image pickupunit 100 according to the first embodiment of the technology.

FIG. 9 is a schematic diagram illustrating a generation example of a 3Dpicture by the image pickup unit 100 according to the first embodimentof the technology.

FIG. 10 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 100 according to the first embodiment of thetechnology.

FIG. 11 is a flowchart illustrating an example of a processing procedureof a stereoscopic picture generating processing (step S920) in the imagepickup processing procedure according to the first embodiment of thetechnology.

FIG. 12 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 400 according to a secondembodiment of the technology.

FIG. 13 is a diagram schematically illustrating a concept of automaticfocus using pixel values of nine pixel circuits in the parallaxdetection pixel 230 according to the second embodiment of thetechnology.

FIG. 14 is a diagram schematically illustrating focus determination byphase difference detection by a focus determination section 410according to the second embodiment of the technology.

FIG. 15 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 400 according to the second embodiment of thetechnology.

FIG. 16 is a flowchart illustrating an example of a processing procedureof focus processing (step S940) in the image pickup processing procedureaccording to the second embodiment of the technology.

FIG. 17 is a diagram schematically illustrating an example of an imagepickup device in which parallax detection pixels are arranged in lineonly in a row direction, as a first modification of the first and secondembodiments of the technology.

FIG. 18 is a diagram schematically illustrating an example of an imagepickup device in which parallax detection pixels are arranged withpredetermined intervals (arranged in an island shape) in a row directionand a column direction, as a second modification of the technology.

FIG. 19 is a diagram schematically illustrating modifications of thecross-sectional structure of the picture generation pixel and theparallax detection pixel, as third to fifth modifications of thetechnology.

FIG. 20 is a schematic diagram illustrating modifications of theparallax detection pixel, as sixth to ninth modifications of thetechnology.

FIG. 21 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 500 according to a thirdembodiment of the technology.

FIG. 22 is a diagram schematically illustrating an example of adiaphragm 510 according to the third embodiment of the technology.

FIG. 23 is a diagram schematically illustrating driving directions of afirst diaphragm 511 and a second diaphragm 515 in the case where thediaphragm 510 according to the third embodiment of the technology isdriven so that only an aperture area is varied while a base-line lengthis fixed.

FIG. 24 is a diagram schematically illustrating the driving directionsof the first diaphragm 511 and the second diaphragm 515 in the casewhere the diaphragm 510 according to the third embodiment of thetechnology is driven so that only the base-line length is varied whilethe aperture area is fixed.

FIG. 25 is a diagram schematically illustrating the case where anaperture part of the diaphragm 510 according to the third embodiment ofthe technology is formed into a shape suitable for capturing a planarpicture.

FIG. 26 is a diagram schematically illustrating a setting screen of apicture to be captured and a setting screen of a 3D intensity that aredisplayed on a display section 151 according to the third embodiment ofthe technology.

FIG. 27 is a diagram schematically illustrating change of an image bythe variation of the base-line length by the diaphragm 510 according tothe third embodiment of the technology.

FIG. 28 is a diagram schematically illustrating a difference between anaperture plane of the diaphragm 510 according to the third embodiment ofthe technology and an aperture plane of an existing diaphragm.

FIG. 29 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 500 according to the third embodiment of thetechnology.

FIG. 30 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 600 according to a fourthembodiment of the technology.

FIG. 31 is a diagram schematically illustrating an example of adiaphragm 610 according to the fourth embodiment of the technology.

FIG. 32 is a diagram schematically illustrating an example of a shape ofan aperture part formed by the diaphragm 610 according to the fourthembodiment of the technology.

FIG. 33 is a diagram schematically illustrating an example of adiaphragm having a simple configuration suitable for capturing a 3Dpicture, as modifications of the third and fourth embodiments of thetechnology.

DESCRIPTION OF EMBODIMENTS

Preferred embodiments for carrying out the technology (hereinafter,referred to as embodiments) will be described below. The descriptionwill be given in the following order.

-   1. First embodiment (image pickup control: an example in which    parallax is detected by parallax detection pixels and a 3D picture    is generated)-   2. Second embodiment (image pickup control: an example in which    pixel values of parallax detection pixels are used to detect a phase    difference)-   3. Modifications-   4. Third embodiment (diaphragm control: an example in which    brightness and a base-line length are independently controlled in    horizontal shooting)-   5. Fourth embodiment (diaphragm control: an example in which    brightness and a base-line length are independently controlled both    in horizontal shooting and vertical shooting)-   6. Modifications

1. First Embodiment Example of Functional Configuration of Image PickupUnit

FIG. 1 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 100 according to a firstembodiment of the technology. The image pickup unit 100 is an imagepickup unit generating 3D picture by a single lens. The image pickupunit 100 captures an image of an object to generate picture data (acaptured picture), and records the generated picture data as 2D picturecontents or 3D picture contents (still picture contents or movingpicture contents). Note that, in the following description, an examplein which still picture contents (still picture file) are recorded aspicture contents (picture file) will be described mainly.

The image pickup unit 100 includes a lens section 110, an operationreceiving section 120, a control section 130, an image pickup device200, a signal processing section 300, a posture detection section 140, adisplay section 151, a memory section 152, and a drive section 170.

The lens section 110 collects light from an object (object light). Thelens section 110 includes a zoom lens 111, a diaphragm 112, and a focuslens 113.

The zoom lens 111 is driven by the drive section 170, and accordinglymoves in an optical axis direction to vary a focal length, therebyadjusting a magnification of an object included in the captured picture.

The diaphragm 112 is a shielding component that varies degree of anaperture by drive of the drive section 170, and accordingly adjusts alight amount of object light entering the image pickup device 200.

The focus lens 113 is driven by the drive section 170, and accordinglymoves in the optical axis direction, thereby adjusting focus.

The operation receiving section 120 receives operation from a user. Forexample, when a shutter button (not illustrated) is pushed, theoperation receiving section 120 supplies a signal related to the push asan operation signal to the control section 130.

The control section 130 controls operation of each section in the imagepickup unit 100. Note that, in FIG. 1, only main signal lines areillustrated and other signal lines are omitted. For example, when theshutter button is pushed and the control section 130 receives anoperation signal for starting still picture recording, the controlsection 130 supplies a signal related to execution of the still picturerecording to the signal processing section 300.

The image pickup device 200 is an image sensor performing photoelectricconversion of the received object light into an electrical signal. Forexample, the image pickup device 200 is configured of an x-y addresstype sensor such as complementary metal oxide semiconductor (CMOS)sensor, a charge coupled device (CCD) sensor, or the like. In the imagepickup device 200, pixels generating a signal for generating a capturedpicture based on the received object light (picture generation pixels)and pixels detecting parallax for generating a 3D picture (parallaxdetection pixels) are arranged. Incidentally, the image pickup device200 will be described with reference to FIG. 2. In addition, the picturegeneration pixel and the parallax detection pixel will be described withreference to FIG. 3 to FIG. 6. The image pickup device 200 supplies theelectrical signal (a picture signal) generated by the photoelectricconversion, to the signal processing section 300 for each frame (picturedata).

The signal processing section 300 performs predetermined signalprocessing on the electrical signal supplied from the image pickupdevice 200. For example, the signal processing section 300 converts theelectrical signal supplied from the image pickup device 200 into adigital electrical signal (a pixel value), and then performs black levelcorrection, defect correction, shading correction, mixed-colorcorrection, and the like. Moreover, the signal processing section 300generates a 3D picture (a stereoscopic picture), based on the electricalsignal subjected to the correction. The signal processing section 300includes, as a functional configuration for generating a 3D picture, a2D picture generation section 310, a parallax detection section 320, anda 3D picture generation section 330. Note that the signal processingsection 300 is an example of a stereoscopic picture generation sectiondescribed in claims.

The 2D picture generation section 310 generates a 2D picture (a planarpicture), based on the electrical signal (the pixel value) of thepicture generation pixel. The 2D picture generation section 310interpolates the electrical signal (the pixel value) corresponding to aposition of the parallax detection pixel, based on the electrical signal(the pixel value) of the picture generation pixel, and performs demosaicprocessing to generate a planar picture. The 2D picture generationsection 310 supplies the generated planar picture to the 3D picturegeneration section 330.

The parallax detection section 320 generates a parallax informationpicture, based on the electrical signal (the pixel value) of theparallax detection pixel. In this case, the parallax information pictureis a picture including information (parallax information) related todifference (parallax) between a left-eye picture and a right-eyepicture. The parallax detection section 320 supplies the generatedparallax information picture to the 3D picture generation section 330.

The 3D picture generation section 330 generates a 3D picture (astereoscopic picture), based on the parallax information picture and the2D picture. The 3D picture generation section 330 generates a left-eyepicture and a right-eye picture as the 3D picture. For example, the 3Dpicture generation section 330 generates the 3D picture by displacing aposition of a captured image of each object in the 2D picture, based onthe parallax information indicated by the parallax information picture.The 3D picture generation section 330 stores, as stereoscopic picturecontents, data of the generated left-eye picture (left-eye picture data)and data of the generated right-eye picture (right-eye picture data) inthe memory section 152. In addition, the 3D picture generation section330 allows the display section 151 to display the left-eye picture dataand the right-eye picture data as the stereoscopic picture contents.

The posture detection section 140 detects a posture (inclination) of theimage pickup unit 100. For example, the posture detection section 140 isconfigured of a gyroscope or an acceleration sensor. The posturedetection section 140 supplies information (posture information) relatedto the detected posture of the image pickup unit 100 to the parallaxdetection section 320.

The display section 151 displays a picture based on the stereoscopicpicture contents supplied from the 3D picture generation section 330.For example, the display section 151 is configured of a color liquidcrystal panel.

The memory section 152 holds the stereoscopic picture contents suppliedfrom the 3D picture generation section 330. For example, removablerecording media (one or more recording media) including disks such as adigital versatile disk (DVD), semiconductor memories such as a memorycard, and the like are used as the memory section 152. In addition, suchrecording media may be incorporated in the image pickup unit 100, or maybe detachable from the image pickup unit 100.

The drive section 170 drives the zoom lens 111, the diaphragm 112, andthe focus lens 113. For example when an instruction for moving the focuslens 113 is supplied from the control section 130, the drive section 170moves the focus lens 113 based on the instruction.

[Arrangement Example of Pixels in Image Pickup Device]

FIG. 2 is a schematic diagram illustrating an example of an arrangementof pixels provided in the image pickup device 200 according to the firstembodiment of the technology. Note that, in the embodiment of thetechnology, in the case where a pixel arrangement on a light receivingsurface of the image pickup device is described (for example, FIG. 2,FIG. 3, and FIG. 7), the case where the arrangement is viewed from abackside of the light receiving surface of the image pickup device isdescribed with schematic illustration, for convenience of description.

The description is made with the assumption of XY-axis in which avertical direction is referred to as a Y-axis, and a lateral directionis referred to as an X-axis in the figure. In addition, in the figure,it is assumed that a lower left corner is referred to as an origin ofthe XY-axis, a direction from bottom to top is referred to as a plusside of the Y-axis, and a direction from left to right is referred to asa plus side of the X-axis. Note that, in the figure, a specificdirection (a direction corresponding to the horizontal direction (thelateral direction) of the captured picture) in the image pickup device200 is referred to as the X-axis direction, and an orthogonal direction(a direction corresponding to the vertical direction of the capturedpicture) orthogonal to the specific direction is referred to as theY-axis direction. Moreover, in the figure, a direction from back tofront is referred to as a plus side of the Z-axis. Note that the Z-axisis an axis parallel to an optical axis, and the plus direction of theZ-axis is a traveling direction of object light from the object towardthe image pickup device. In addition, a reading direction of a signal inthe image pickup device 200 is the X-axis direction (read line by line),and a long side direction of the image pickup device 200 is the X-axisdirection, and a short side direction thereof is the Y-axis direction.

In (a) of the figure, for convenience of description, the pixelarrangement is described with use of a region (region 210) for a part ofpixels configuring the image pickup device 200. Note that thearrangement of the pixels in the image pickup device 200 is anarrangement in which a pixel arrangement illustrated in the region 210is used as one unit, and a pixel arrangement corresponding to the unit(the pixel arrangement corresponding to the region 210) is repeated inthe X-axis direction and the Y-axis direction. Note that, in (b) of thefigure, a region (a region 250) in which the pixel arrangementillustrated in the region 210 is repeated in the X-axis direction andthe Y-axis direction is schematically illustrated.

In (a) of the figure, the pixel arrangement of the picture generationpixels and the parallax detection pixels in a partial region (the region210) of the image pickup device 200.

In the figure, one pixel circuit is indicated by one square (a smallestsquare in (a) of the figure). Incidentally, in the first embodiment ofthe technology, as for the picture generation pixel, one squareindicates one picture generation pixel because one pixel circuitconfigures one pixel.

In the image pickup device 200, as the picture generation pixels, apixel (R pixel) receiving red (R) light by a color filter that allowsred light to pass therethrough, a pixel (G pixel) receiving green (G)light by a color filter that allows green light to pass therethrough arearranged. Moreover, in the image pickup device 200, in addition to the Rpixel and the G pixel, a pixel (B pixel) receiving blue (B) light by acolor filter that allows blue light to pass therethrough is arranged asthe picture generation pixel. In the region 210, the R pixel, the Gpixel, and the B pixel are indicated by a dotted square, a white square,and a gray square, respectively (for example, see the pixels in a region220).

The parallax detection pixel 230 is a pixel detecting parallax forgenerating a 3D picture. The parallax detection pixel 230 is configuredof nine pixel circuits. The parallax detection pixel 230 is providedwith one microlens covering the nine pixel circuits. Incidentally, thepicture generation pixel is not provided with a microlens covering thenine pixels. In the parallax detection pixel 230, the nines pixelcircuits are uniformly covered with a color filter that allows green (G)light to pass therethrough. In the region 210, the parallax detectionpixel 230 is indicated in such a manner that the nine pixel circuits(nine smallest squares) are enclosed by a heavy-line square, and oneheavy-line circle (indicating a microlens) is added in the heavy-linesquare. In other words, the parallax detection pixels 230 are arrangedin three rows in the middle of rows of the pixel circuits in the region210 (a row R1 in (a) of the figure) and in three columns in the middleof columns of the pixel circuits (see a column C1 in (a) of the figure).

Incidentally, the picture generation pixel will be described using aregion 225 in (a) of the figure with reference to (a) of FIG. 3. Inaddition, the parallax detection pixel will be described with referenceto (b) of FIG. 3. Moreover, the cross-sectional structure of the picturegeneration pixel and the parallax detection pixel will be described withreference to (b) of FIG. 4.

In (b) of FIG. 2, a region in which the region 210 illustrated in (a) ofthe figure is repeated in the X-axis direction and the Y-axis directionis illustrated. The pixel arrangement in the image pickup device 200 isformed by repeating the region illustrated in (a) of the figure in theX-axis direction and the Y-axis direction. Therefore, as illustrated byheavy lines (parallax detection pixel lines 234) in the region 250 in(b) of the figure, the parallax detection pixels 230 are arranged inline in the row direction and the column direction (in a lattice) of theimage pickup device 200 in an arbitrary pixel cycle.

[Example of Picture Generation Pixel and Parallax Detection Pixel]

FIG. 3 is a schematic diagram illustrating an example of the picturegeneration pixel and the parallax detection pixel that are provided inthe image pickup device 200 according to the first embodiment of thetechnology.

In (a) of the figure, the nine picture generation pixels in the region225 in (a) of FIG. 2 are illustrated. In (a) of the figure, the picturegeneration pixels (the R pixel, the G pixel, and the B pixel) areindicated with symbols (R, G, and B) and patterns (the R pixel isindicated as a gray region, the G pixel is indicated as a white region,and the B pixel is indicated as a dotted region). Incidentally, as forthe G pixel, the G pixel in a row (line) including the R pixel (an Rpixel 226) is indicated as a Gr pixel 227, and the G pixel in a row(line) including the B pixel (a B pixel 229) is indicated as a Gb pixel228.

In addition, in (a) of the figure, the microlens (a micro lens 261)arranged for each picture generation pixel is indicated by a dashedcircle. The image pickup device 200 is provided with two microlenslayers for forming two types of microlenses, and the microlens 261 isformed of a microlens layer close to a color filter layer. Note that asectional diagram of the picture generation pixel is described withreference to (b) of FIG. 4, and thus the detailed description thereof isomitted here.

As illustrated in the region 225 (see also (a) of FIG. 2), the picturegeneration pixels are arranged based on Bayer arrangement. In addition,each picture generation pixel is provided with one microlens and iscovered therewith.

(b) of FIG. 3 illustrates the parallax detection pixel (the parallaxdetection pixel 230). In the parallax detection pixel 230 illustrated in(b) of FIG. 3, the nine pixel circuits in the parallax detection pixel230 are indicated by squares with numbers of 1 to 9 (referred to aspixel circuits 1 to 9). All of the pixel circuits 1 to 9 are uniformlycovered with the color filter that allows green (G) light to passtherethrough.

In addition, in the parallax detection pixel 230, the microlens (themicrolens 261) provided for each pixel circuit is indicated by a dashedcircle. The microlens 261 in the parallax detection pixel 230 is similarto the microlens (the microlens 261) of the picture generation pixel,and is formed of a microlens layer close to a color filter layer.

Further, the parallax detection pixel 230 is provided with one largemicrolens that covers all of the nine microlenses 261 of the parallaxdetection pixel 230 (covers all of the pixel circuits 1 to 9). In (b) ofthe figure, the large microlens (a parallax-detection-use microlens 231)is indicated by a heavy-line circle.

As illustrated in (b) of the figure, in the parallax detection pixel230, one large microlens is arranged so as to cover the nine pixelcircuits. In addition, in the parallax detection pixel 230, the colorfilter that allows green (G) light to pass therethrough uniformly coversthe nine pixel circuits.

[Example of Cross-Sectional Structure of Picture Generation Pixel andParallax Detection Pixel]

FIG. 4 is a schematic diagram illustrating an example of across-sectional structure of the picture generation pixel and theparallax detection pixel according to the first embodiment of thetechnology.

In (a) of the figure, to describe a position of a cross-sectionalsurface in the cross-sectional structure illustrated in (b) of thefigure, a pixel circuit of three rows and nine columns viewed from thebackside of the light receiving surface of the image pickup device 200and a position of the cross-sectional surface (a-b line) in (b) of thefigure are illustrated. In (b) of the figure, as illustrated in (a) ofthe figure, description will be given on the assumption that the pixelcircuits arranged in three columns in the middle of the pixel circuitsof three rows and nine columns are the parallax detection pixels.

In (b) of the figure, a cross-sectional structure taken along the a-bline in (a) of the figure is illustrated. In (b) of the figure, themicrolenses 261, R filters 262, G filters 263, photodetectors 264,wirings 269, the parallax detection pixel 230, theparallax-detection-use microlens 231, and a G filter 232 areillustrated. In addition, in (b) of the figure, a microlens layer S1 anda parallax-detection-use microlens layer S2 are illustrated.

The microlens 261 is a lens collecting object light in thephotodetector. The microlens 261 is provided for each pixel circuit.Moreover, the microlens 261 is provided on the microlens layer S1 (alayer close to the photodetector 264) of the two microlens layers (themicrolens layer S1 and the parallax-detection-use microlens layer S2)provided in the image pickup device.

The photodetector 264 converts the received light into an electricalsignal (photoelectric conversion), thereby generating an electricalsignal having an intensity corresponding to an amount of the receivedlight. For example, the photodetector 264 is configured of a photo diode(PD). The photodetector 264 is provided for each pixel circuit. In otherwords, nine photodetectors are provided in the parallax detection pixel230 that is configured of the nine pixel circuits.

The R filter 262, the G filter 263, and the G filter 232 are colorfilters allowing light with a specific wavelength range to passtherethrough. The R filter 262 is a color filter allowing light within awavelength range showing red (R) to pass therethrough, and allows thephotodetector of the R pixel to receive light within a wavelength rangeshowing red. In addition, the G filter 263 and the G filter 232 arecolor filters allowing light within a wavelength range showing green (G)to pass therethrough. The G filter 263 allows the photodetector of the Gpixel to receive light within a wavelength range showing green, and theG filter 232 allows the photodetector of the parallax detection pixel230 to receive light within a wavelength range showing green. In thepicture generation pixel, a color filter (the R filter 262, the G filter263, or the B filter) corresponding to light within a wavelength range(R, G, or B) that is received by the picture generation pixel isprovided. Moreover, in the parallax detection pixel, the G filter 263covers all of the nine pixel circuits of the parallax detection pixel.

The wiring 269 connects each circuit in each pixel circuit. For example,the wiring connecting each circuit is arranged in such a manner thatthree wirings are layered with respect to the optical axis, as with thewiring 269 illustrated in (b) of the figure. In addition, the wiring 269is formed of a metal, and thus functions as a light shielding layer thatshields object light leaking into adjacent pixels. Moreover, the wiring269 is arranged at an end of each pixel circuit so as not to block lightentering the photodetector.

The parallax-detection-use microlens 231 is a lens collecting objectlight to detect parallax. The parallax-detection-use microlens 231 isformed in a layer (the parallax-detection-use microlens layer S2)farther from the photodetector of the two microlens layers. In otherwords, in the parallax detection pixel 230, the parallax-detection-usemicrolens 231 is provided on the microlens (the microlens 261) of theparallax detection pixel 230 (on a minus side in the Z-axis direction)so as to cover the microlenses 261. Note that a microlens is notprovided at a position of the picture generation pixel in theparallax-detection-use microlens layer S2, and the surface is flat notblocking passing light.

Next, a relationship between the pixel circuit and the object lightentering the parallax-detection-use microlens in the parallax detectionpixel will be described with reference to FIG. 5.

[Example of Object Light Entering Parallax Detection Pixel]

FIG. 5 is a diagram schematically illustrating object light that isreceived by the parallax detection pixel according to the firstembodiment of the technology.

In the figure, a cross-sectional structure of the parallax detectionpixel 230 is illustrated in (b) of the figure, and an exit pupil (anexit pupil E1) that is a shape of a diaphragm viewed from the parallaxdetection pixel 230 is schematically illustrated in (a) of the figure.Note that the exit pupil essentially has a substantially circular shape,however in (a) of the figure, the exit pupil having a short length inthe Y-axis direction (elliptical shape) is illustrated for convenienceof description. In addition, the exit pupil viewed from the image pickupdevice 200 side is illustrated.

Incidentally, the cross-sectional structure of the parallax detectionpixel 230 illustrated in (b) of the figure is the same as thecross-sectional structure of the parallax detection pixel 230illustrated in (b) of FIG. 4. In addition, in (b) of FIG. 4, thephotodetector of the pixel circuit 4 (see (b) of FIG. 3) is indicated asa photodetector (4) 291, the photodetector of the pixel circuit 5 isindicated as a photodetector (5) 292, and the photodetector of the pixelcircuit 6 is indicated as a photodetector (6) 293.

Furthermore, in (b) of the figure, part of the object light received bythe photodetector (4) 291 is indicated as a region (a region R23) with alot of dots. In addition, part of the object light received by thephotodetector (5) 292 is indicated as a gray region (a region R22).Further, part of the object light received by the photodetector (6) 293is indicated as a region (a region R21) with small number of dots.

Moreover, in the exit pupil E1 illustrated in (a) of the figure, aregion indicated as a region 21 in (b) of the figure, where the objectlight (the object light to be received by the pixel circuit 6) passes,is indicated as a region (a region R11) with small number of dots in theexit pupil E1. Likewise, a region indicated as a region 22, where theobject light (the object light to be received by the pixel circuit 5)passes, is indicated as a gray region (a region R12) in the exit pupilE1. A region indicated as a region 23, where the object light (theobject light to be received by the pixel circuit 4) passes, is indicatedas a region (a region R13) with a lot of dots in the exit pupil E1.Further, in the exit pupil E1, regions where the object light to bereceived by other pixel circuits passes, are indicated by regions 14 to19.

A relationship between the object light received by the photodetector(4) 291 to the photodetector (6) 293 and the regions R11 to R13 in theexit pupil E1 is described now. The parallax-detection-use microlens 231functions to collect the object light so that the object light that haspassed through the specific regions (the regions corresponding to therespective photodetectors) in the exit pupil E1 is received by therespective photodetectors of the pixel circuits 1 to 9. As a result, thephotodetector of the pixel circuit 4 (the photodetector (4) 291)receives the object light that has passed through the region R13. Thephotodetector of the pixel circuit 5 (the photodetector (5) 292)receives the object light that has passed through the region R12. Inaddition, the photodetector of the pixel circuit 6 (the photodetector(6) 293) receives the object light that has passed through the regionR11. Note that the same applies to the not-illustrated pixel circuits 1to 3 and the not-illustrated pixel circuits 7 to 9. The photodetector ofthe pixel circuit 1 receives the object light that has passed throughthe region 19, the photodetector of the pixel circuit 2 receives theobject light that has passed through the region 18, and thephotodetector of the pixel circuit 3 receives the object light that haspassed through the region 17. Moreover, the photodetector of the pixelcircuit 7 receives the object light that has passed through the region16, the photodetector of the pixel circuit 8 receives the object lightthat has passed through the region 15, and the photodetector of thepixel circuit 9 receives the object light that has passed through theregion 14.

[Example of Principle of Parallax Detection by Parallax Detection Pixel]

FIG. 6 is a diagram schematically illustrating a principle of parallaxdetection by the parallax detection pixel 230 according to the firstembodiment of the technology.

In (a) and (c) of the figure, an imaging position on an imaging surfaceof the object light (corresponding to the object light received by aleft eye of a user) that passes through the left side of the exit pupil(the exit pupil E1) is schematically illustrated. In addition, in (b)and (d) of the figure, an imaging position on the imaging surface of theobject light (corresponding to the object light received by a right eyeof the user) that passes through the right side of the exit pupil (theexit pupil E1) is schematically illustrated.

In (a) and (b) of the figure, an object that is focused on (a focusedobject) is indicated by a black rectangle (a focused object 271) locatedabove the exit pupil E1 (on the minus side than the exit pupil E1 in theZ-axis direction). In addition, an object (a close positioned object)located closer to the image pickup unit 100 compared with the focusedobject 271 is indicated by a dotted circle (a close positioned object272) located at a position above the exit pupil E1 and lower than thefocused object 271. Note that, it is assumed that the focused object 271and the close positioned object 272 are located on a line (a chainedline in (a) and (b) of the figure) that passes through the center of theexit pupil E1 and is parallel to the optical axis of the lens, forconvenience of description.

Moreover, an optical path of the light from the focused object 271 andthe close positioned object 272 is indicated by a line (a solid line (aline L1 or a line L3) for the focused object 271, and a dashed line (aline L2 or a line L4) for the close positioned object 272) that passesthrough the exit pupil E1 and extends to the imaging surface. Then, theimaging positions on the imaging surface of the focused object 271 andthe close positioned object 272 are indicated by the black rectangle andthe dotted circle, respectively, that are located at positions where thelines L1 to L4 and the imaging surface are intersected with each other.Note that, as for the light from the close positioned object 272, theimaging position in the case of the assumption that the light is focusedon the close positioned object 272 is schematically illustrated by adashed circle located at a position where the dashed line (the line L2or the line L4) and the chained line are intersected with each other.

In addition, (c) and (d) of FIG. 6 schematically illustrate images(pictures 281 and the 282) when the imaging surfaces illustrated in (a)and (b) of the figure are viewed from the backside (on a side oppositeto the light receiving surface of the image pickup device 200). In thepicture 281, the black rectangle and the dotted circle are illustrated,and a distance (−ΔX) between the black rectangle and the dotted circleis also illustrated. Likewise, in the picture 282, the black rectangle,the dotted circle, and a distance (+ΔX) therebetween are illustrated.

The imaging position of the object light that passes through the leftside of the exit pupil E1 and the imaging position of the object lightthat passes through the right side thereof are described now withreference to (a) to (d) of the figure.

First, the optical path (the line L1 and the line L3) of the light fromthe focused object 271, which illustrates the optical path in the casewhere focus is adjusted (focusing) is described. In the case where theobject to be captured is focused, the object light that has passedthrough the exit pupil E1 enters (is collected to) the position of theimaging surface corresponding to the position of the object,irrespective of the position in the exit pupil E1 where the object lightpasses. In other words, the imaging position of the light from thefocused object 271 that passes through the left side of the exit pupilE1, is the same as the imaging position of the light from the focusedobject 271 that passes through the right side of the exit pupil E1.

On the other hand, when the object to be captured is out of focus, thelight-entering position on the imaging surface differ depending ondifference of the position in the exit pupil E1 where the object lightpasses. Since light that is originally collected on a surface differentfrom an imaging surface (light-collected position is indicated by adashed circle below the imaging surface in (a) and (b) of the figure) isreceived by the imaging surface, the light-entering position on theimaging surface is deviated depending on the degree of deviation of thefocus. As illustrated by the close positioned object 272, the line L2,and the line L4 in (a) and (b) of the figure, when the object to becaptured is located on a side closer to the lens than the front focusingsurface (the surface at which the focused object 271 is located), theback focusing surface (the surface at which the dashed circle islocated) is located behind the imaging surface (on the plus side in theZ-axis direction). In other words, the light (the line L2) that is fromthe close positioned object 272 and enters the imaging surface throughthe left side of the exit pupil E1, enters the imaging surface at theposition shifted to the left from the position where the light from thefocused object 271 enters (see (a) of the figure). In addition, thelight (the line L4) that is from the close positioned object 272 andenters the imaging surface through the right side of the exit pupil E1,enters the imaging surface at the position shifted to the right from theposition where the light from the focused object 271 enters (see (b) ofthe figure).

In this way, when the object that is out of focus is captured, theentering position on the imaging surface is different between the lightfrom the object that has passed through the left side of the exit pupiland the light from the object that has passed through the right sidethereof, depending on the degree of deviation of the focus. Thedeviation causes defocusing, thereby resulting in blur in the picturegenerated from the signal of the picture generation pixel. On the otherhand, in the parallax detection pixel 230, the light that has passedthrough the left side of the exit pupil is received by thephotodetectors of the pixel circuits in the right column (the pixelcircuits 3, 6, and 9 in (b) of FIG. 3) of (the nine pieces of) the pixelcircuits of three rows and three columns. In addition, the light thathas passed through the right side of the exit pupil is received by thephotodetectors of the pixel circuits in the left column (the pixelcircuits 1, 4, and 6 in (b) of FIG. 3).

In other words, the information of the left-eye image is allowed to begenerated by the signals from the pixel circuits in the right column(third column) in the parallax detection pixel 230, as illustrated in(a) and (c) of the figure. In addition, the information of the right-eyeimage is allowed to be generated by the signals from the pixel circuitsin the left column (first column) in the parallax detection pixel 230,as illustrated in (b) and (d) of the figure. As described above, whenthe pictures are generated based on the signals from the parallaxdetection pixel 230, two pictures in which images are displaceddepending on the distance of the object are allowed to be generated asillustrated in the pictures 281 and 282.

[Example of Parallax Detection Direction in Horizontal Shooting]

FIG. 7 is a diagram schematically illustrating an example of a directionof the parallax detection by the parallax detection pixels 230 when theimage pickup unit 100 according to the first embodiment of thetechnology is used to perform the horizontal shooting.

(a) of the figure illustrates a posture of the image pickup unit 100assumed in the figure. In the figure, as illustrated in (a) of thefigure, it is assumed that the image pickup unit 100 is used sideways(the lateral direction corresponds to the X-axis direction, and thevertical direction corresponds to the Y-axis direction) to performshooting (the horizontal shooting) so that the long side of the picturecorresponds to the horizontal direction and the short side thereofcorresponds to the vertical direction.

Note that the case where the image pickup unit 100 is rotated by 90degrees in a counterclockwise direction and the shooting (the verticalshooting) is performed using the image pickup unit 100 lengthwise willbe described with reference to FIG. 8.

(b) of the figure illustrates the posture of the image pickup device 200in the case of the horizontal shooting and lines (parallax detectionlines (row direction) 235) indicating the parallax detection pixels 230used in the parallax detection in the posture. Note that in (b) of thefigure, the region 250 illustrated in (b) of FIG. 2 is illustrated as itis so as to illustrate arrangement of the pixels in the image pickupdevice 200.

In (b) of the figure, the image pickup device 200 in which the long sidedirection (the X-axis direction) corresponds to the lateral (horizontal)direction, and the short side direction (the Y-axis direction)corresponds to the vertical direction is illustrated. In addition, inthe image pickup unit 200, lines in which the parallax detection pixels230 used in the parallax detection in the horizontal shooting, out ofthe parallax detection pixels 230 arranged in lines in the row directionand the column direction are indicated by thick solid lines (parallaxdetection pixel lines (row direction) 235).

As illustrated in (b) of the figure, in the case of the horizontalshooting, the parallax detection pixels 230 arranged in lines in the rowdirection of the image pickup device 200 are used to detect parallax.Accordingly, the parallax in the horizontal (lateral) direction isallowed to be detected.

In (c) of the figure, the posture of the parallax detection pixel 230 inthe case of the horizontal shooting is illustrated. Moreover, in theparallax detection pixel 230, the pixel circuits generating signalsrelated to a right eye are indicated by rectangles with a lot of dots(the pixel circuits 1, 4, and 7). Likewise, the pixel circuitsgenerating information related to a left eye are indicated by rectangleswith small number of dots (the pixel circuits 3, 6, and 9).Incidentally, in (c) of the figure, the dashed circle (the microlens261) illustrated in (b) of FIG. 3 is omitted for convenience ofdescription.

As illustrated in (c) of FIG. 7, in the case of the horizontal shooting,the parallax detection pixels 230 arranged in the row direction of theimage pickup device 200 are used to detect parallax.

[Example of Parallax Detection Direction in Vertical Shooting]

FIG. 8 is a diagram schematically illustrating an example of a directionof the parallax detection by the parallax detection pixels 230 when theimage pickup unit 100 according to the first embodiment of thetechnology is used to perform the vertical shooting.

(a) of the figure illustrates a posture of the image pickup unit 100assumed in the figure. The image pickup unit 100 in (a) of the figure isin a posture in which the image pickup unit 100 in (a) of FIG. 7 isrotated by 90 degrees in a counterclockwise direction. As describedabove, in FIG. 8, it is assumed that the shooting (the verticalshooting) is performed while the image pickup unit 100 is usedlengthwise (the lateral direction corresponds to the Y-axis direction,and the vertical direction corresponds to the X-axis direction) asillustrated in (a) of the figure.

(b) of the figure illustrates the posture of the image pickup device 200in the case of the vertical shooting and lines (parallax detection lines(column direction) 236) indicating the parallax detection pixels 230used in the posture.

In (b) of the figure, the image pickup device 200 in which the long sidedirection (the X-axis direction) corresponds to the vertical direction,and the short side direction (the X-axis direction) corresponds to thelateral (horizontal) direction is illustrated. The image pickup unit 200in (b) of the figure is in a state where the image pickup device 200 in(b) of FIG. 7 is rotated by 90 degrees in a counterclockwise direction.In addition, in the image pickup unit 200 in (b) of FIG. 8, lines inwhich the parallax detection pixels 230 used in the parallax detectionin the vertical shooting, out of the parallax detection pixels 230arranged in lines in the row direction and the column direction areindicated by thick solid lines (parallax detection pixel lines (columndirection) 236).

As illustrated in (b) of the figure, in the case of the verticalshooting, the parallax detection pixels 230 arranged in lines in thecolumn direction of the image pickup device 200 are used to detectparallax. Accordingly, the parallax in the horizontal (lateral)direction is allowed to be detected even in the case where the imagepickup unit 100 is used lengthwise (the long side of the picturecorresponds to the vertical direction, and the short side thereofcorresponds to the lateral direction).

In (c) of the figure, the posture of the parallax detection pixel 230 inthe case of the vertical shooting is illustrated. The parallax detectionpixel 230 in (c) of the figure is in a state where the parallaxdetection pixel 230 in (c) of FIG. 7 is rotated by 90 degrees in thecounterclockwise direction. Moreover, in the parallax detection pixel230 in (c) of FIG. 8, the pixel circuits generating signals related tothe right eye are indicated by rectangles with a lot of dots (the pixelcircuits 1 to 3). Likewise, the pixel circuits generating informationrelated to the left eye are indicated by rectangles with small number ofdots (the pixel circuits 7 to 9).

In this way, in the case of the vertical shooting, parallax is detectedusing the parallax detection pixels arranged in lines in a columndirection of the image pickup device 200. Accordingly, similarly to thecase of the horizontal shooting, information corresponding to the lefteye and the right eye in the case where a human stands upright(information of horizontal parallax) is allowed to be generated.

[Generation Example of 3D Picture]

FIG. 9 is a schematic diagram illustrating a generation example of a 3Dpicture by the image pickup unit 100 according to the first embodimentof the technology.

In the figure, pictures that are generated by the 2D picture generationsection 310, the parallax detection section 320, and the 3D picturegeneration section 330 based on the picture data generated by the imagepickup device 200 capturing an image of the object are illustrated. Inaddition, in the figure, with use of the pictures, a flow to generate astereoscopic picture (a left-eye picture and a right-eye picture) basedon the picture data generated by the image pickup device 200 isdescribed in order. Note that, in the figure, it is assumed that theshooting is performed by the horizontal shooting as illustrated in FIG.7.

First, generation of the 2D picture (the planar picture) by the 2Dpicture generation section 310 is described with reference to (a) and(b) of the figure.

In (a) of the figure, a planar picture (a planar picture 311) that isgenerated by the 2D picture generation section 310 based on the signalsgenerated by the picture generation pixels in the image pickup device200, before interpolation of the signal of the parallax detection pixelis illustrated. In the planar picture 311, two persons (a person 351 anda person 352) as captured objects are shown. In the figure, it isassumed that the shooting was performed in a state where the person 352was focused. In addition, in the figure, it is assumed that the person351 is positioned closer to the lens compared with the person 352. Inother words, it is assumed that the person 351 is out of focus. Dashedlines enclosing the person 351 fourfold schematically illustrate blur ofthe image due to defocusing.

Moreover, in the planar picture 311, absence of the signals forgenerating picture (absence of the pixel value) at the position wherethe parallax detection pixels are arranged is indicated by a pluralityof gray lines (pixel value missing lines 353) that indicate absence ofthe data (the pixel value) in the planar picture 311.

As described above, the picture (the planar picture 311) in which thepixel values of the positions of the parallax detection pixels aremissed is generated from the signals generated by the picture generationpixels. Therefore, the 2D picture generation section 310 interpolatesthe pixel values at the positions of the parallax detection pixels togenerate a planar picture in which the missing pixel values have beeninterpolated.

In (b) of FIG. 9, a picture (a planar picture 312) after the 2D picturegeneration section 310 performs the interpolation processing on theplanar picture 311 illustrated in (a) of the figure is illustrated. The2D picture generation section 310 performs the interpolation processingon the planar picture 311 to interpolate the missing pixel values (thepixel value missing lines 353 in (a) of the figure). Further, otherinterpolation processing and demosaic processing are performed togenerate the planar picture (the planar picture 312) as with the picturecaptured by an image pickup device (a typical image pickup device) thathas picture generation pixels and no parallax detection pixel. Then, thegenerated planar picture 312 is supplied to the 3D picture generationsection 330.

Next, generation of a parallax information picture by the parallaxdetection section 320 is described with reference to (c) and (d) of thefigure.

In (c) of the figure, two pictures (a left-eye information picture 321and a right-eye information picture 322) that are generated by theparallax detection section 320 based on the signals generated by theparallax detection pixels in the image pickup device 200 and are sourcesof the parallax information picture are illustrated.

The left-eye information picture 321 is a picture generated based on thesignals from the pixel circuits that have received the object lightcorresponding to the light to be received by a left eye of a user, outof the nine pixel circuits in the parallax detection pixel. In addition,the right-eye information picture 322 is a picture generated based onthe signals from the pixel circuits that have received the object lightcorresponding to the light to be received by a right eye of the user,out of the nine pixel circuits in the parallax detection pixel.

Moreover, in the left-eye information picture 321, persons 361 and 362that correspond to the persons 351 and 352, respectively, shown in (a)of the figure are shown. Likewise, in the right-eye information picture322, persons 363 and 364 that correspond to the persons 351 and 352,respectively, shown in (a) of the figure are shown.

The generation of the left-eye information picture 321 and the right-eyeinformation picture 322 by the parallax detection section 320 isdescribed now.

The parallax detection section 320 determines the pixel circuits thathave generated the signals to be the left-eye information picture 321and the pixel circuits that have generated the signals to be theright-eye information picture 322, based on the posture informationsupplied from the posture detection section 140. Since the picture iscaptured by the horizontal shooting in the figure, the parallaxdetection section 320 generates the left-eye information picture 321from the signals of the pixel circuits in the right column (the pixelcircuits 3, 6, and 9 in (b) of FIG. 7), as illustrated in FIG. 7. Inaddition, the parallax detection section 320 generates the right-eyeinformation picture 322 from the signals of the pixel circuits in theleft column (the pixel circuits 1, 4, and 7 in (b) of FIG. 7). Then, theparallax detection section 320 generates the parallax informationpicture, based on the generated pictures (the left-eye informationpicture 321 and the right-eye information picture 322).

In (d) of FIG. 9, the parallax information picture (a parallaxinformation picture 323) generated based on the left-eye informationpicture 321 and the right-eye information picture 322 is schematicallyillustrated.

In the parallax information picture 323, an image (a person 371)including the parallax that is detected based on the person 361 in theleft-eye information picture 321 and the person 363 in the right-eyeinformation picture 322 is shown. Likewise, in the parallax informationpicture 323, an image (a person 372) including the parallax that isdetected based on the person 362 in the left-eye information picture 321and the person 364 in the right-eye information picture 322 is shown.

The parallax information picture 323 is now described focusing on thetwo images (the person 371 and the person 372) shown in the parallaxinformation picture 323.

The person 371 is shown in the parallax information picture 323 in sucha manner that the person 361 in the left-eye information picture 321 andthe person 363 in the right-eye information picture 322 are overlappedwith each other (images of the two persons are overlapped and slightlyshifted to right and left). A distance (a distance 373) between twodashed lines that are extended in the vertical direction from the person371 indicates a distance between the two images that are overlapped andslightly shifted to right and left. In the figure, the object indicatedby the person 371 is out of focus (see the person 351 in (a) and (b) ofthe figure). In other words, the parallax occurs as illustrated in FIG.6, and positional displacement according to the defocusing amount isgenerated between the person 361 in the left-eye information picture 321and the person 363 in the right-eye information picture 322. Thedistance 373 in (d) of FIG. 9 schematically illustrates the positionaldisplacement (the parallax).

On the other hand, the person 372 is shown in the parallax informationpicture 323 in such a manner that the person 362 in the left-eyeinformation picture 321 and the person 364 in the right-eye informationpicture 322 exactly coincide with each other (like one image of aperson). In the figure, since the object indicated by the person 372(the person 362 and the person 364) is focused (see the person 352 in(a) and (b) of the figure), the parallax related to the image of theperson is absent (the parallax is “zero”). Specifically, the position ofthe person 362 in the left-eye information picture 321 and the positionof the person 364 in the right-eye information picture 322 coincide witheach other. In other words, (d) of the figure illustrates absence of theinformation related to the parallax for the image of the person 372 inthe parallax information picture 323.

The parallax information picture 323 illustrated in (d) of the figure isgenerated and then supplied to the 3D picture generation section 330.

Next, generation of a stereoscopic picture (the left-eye picture and theright-eye picture) by the 3D picture generation section 330 is describedwith reference to (e) of the figure.

In (e) of the figure, a left-eye (L) picture (a left-eye picture 331)and a right-eye (R) picture (a right-eye picture 332) that are generatedby the 3D picture generation section 330, based on the planar picture312 and the parallax information picture 323, are schematicallyillustrated. Incidentally, as a method of generating a stereoscopicpicture based on a planar picture and a parallax information picture,various kinds of methods are considerable, and in this case, an examplein which a left-eye picture and a right-eye picture are generated bydisplacing positions of objects in the planar picture based on theparallax information picture is described as an example.

The left-eye picture 331 is a picture to be displayed with respect to aleft eye of a user viewing a captured picture. In the left-eye picture331, a person 382 is shown at a position same as the position of theperson 352 in the planar picture 312 and as the position of the person372 in the parallax information picture 323. In addition, in theleft-eye picture 331, a person 381 is shown at a position same as theright position of the person 371 in the parallax information picture 323(as the position of the person 361 in the left-eye information picture321). The person 381 is an image obtained by displacing the position ofthe person 351 in the planar picture 312. Incidentally, in (b) of thefigure, a dashed line (a line L11) that extends from the center of theperson 381 in the vertical direction is shown in order to schematicallyillustrate the position of the person 381.

The right-eye picture 332 is a picture to be displayed with respect to aright eye of the user viewing the captured picture. In the right-eyepicture 332, a person 384 is shown at a position same as the position ofthe person 352 in the planar picture 312 and as the position of theperson 372 in the parallax information picture 323. In addition, in theright-eye picture 332, a person 383 is shown at a position same as theleft position of the person 371 in the parallax information picture 323(as the position of the person 363 in the right-eye information picture322). The person 383 is an image obtained by displacing the position ofthe person 351 in the planar picture 312. Incidentally, in (b) of thefigure, a dashed line (a line L12) that extends from the center of theperson 383 in the vertical direction is shown in order to schematicallyillustrate the position of the person 381, and an arrow indicating adistance (the distance 373) between the line L11 and the line L12 isalso shown. In other words, the person 383 is shown at a positiondisplaced from the person 381 in the left-eye picture 331 to left sideby about the distance 373.

The 3D picture generation section 330 generates a stereoscopic picture(a 3D picture) that is configured of the left-eye picture 331 and theright-eye picture 332 illustrated in (e) of the figure, based on theplanar picture 312 and the parallax information picture 323.Specifically, the 3D picture generation section 330 displaces theposition of each object in the planar picture 312 according to theparallax indicated by the parallax information picture 323, therebygenerating the left-eye picture 331 and the right-eye picture 332. Sincethere is no parallax information for the person 372 in the parallaxinformation picture 323, the 3D picture generation section 330 generatesthe left-eye picture 331 and the right-eye picture 332 while maintainingthe position of the person 352 in the planar picture 312 (the person 382and the person 384). Moreover, the 3D picture generation section 330displaces the position of the person 351 in the planar picture 312according to the parallax indicated by the person 371 in the parallaxinformation picture 323 (the person 381 and the person 383), therebygenerating the left-eye picture 331 and the right-eye picture 332. The3D picture generation section 330 supplies the generated stereoscopicpicture (the left-eye picture 331 and the right-eye picture 332) asstereoscopic picture contents to the display section 151 and the memorysection 152.

As described above, the signal processing section 300 generates thestereoscopic picture, based on the planar picture and the parallaxinformation picture.

[Operation Example of Image Pickup Unit]

Next, an operation of the image pickup unit 100 according to the firstembodiment of the technology is described with reference to figures.

FIG. 10 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 100 according to the first embodiment of thetechnology.

First, the control section 130 determines whether a starting instructionof image pickup operation for capturing a stereoscopic picture is issuedby a user (step S901). Then, when it is determined that the startinginstruction of the image pickup operation for capturing the picture isnot issued by the user (step S901), the image pickup processingprocedure is ended.

On the other hand, when it is determined that the starting instructionof the image pickup operation for capturing the stereoscopic picture isissued by the user (step S901), a live view picture is displayed on thedisplay section 151 in order to determine the composition of thestereoscopic picture (step S902). Subsequently, the control section 130determines whether a shutter button is pushed halfway by the user whohas determined the composition of the stereoscopic picture (step S903).Then, when it is determined that the shutter button is not pushedhalfway (step S903), the procedure proceeds to step S908.

On the other hand, when it is determined that the shutter button ispushed halfway (step S903), focusing processing in which the focus lens113 is driven to focus on a target to be focused (an object to befocused) is performed (step S904). Subsequently, the control section 130determines whether the shutter button is fully pushed by the user (stepS903). Then, when it is determined that the shutter button is not fullypushed (step S905), the procedure returns to step S902.

On the other hand, when it is determined that the shutter button isfully pushed (step S905), the image pickup device 200 captures an imageof the object (step S906). Then, the signal processing section 300performs stereoscopic picture generation processing for generating astereoscopic picture (a 3D picture) based on the picture data of thecaptured image (step S920). Note that the stereoscopic picturegeneration processing (step S920) will be described with reference toFIG. 11.

Subsequently, the generated stereoscopic picture is stored in the memorysection 152 (step S907). Then, the control section 130 determineswhether the ending instruction of the image pickup operation forcapturing a stereoscopic picture is issued by the user (step S908).Then, when it is determined that the ending instruction of the imagepickup operation for capturing a stereoscopic picture is not issued bythe user (step S908), the procedure returns to step S902.

On the other hand, when it is determined that the ending instruction ofthe image pickup operation for capturing a stereoscopic picture isissued by the user (step S908), the image pickup processing procedure isended.

FIG. 11 is a flowchart illustrating an example of a processing procedureof the stereoscopic picture generation processing (step S920) in theimage pickup processing procedure according to the first embodiment ofthe technology.

First, the 2D picture generation section 310 generates the planarpicture (the 2D picture) based on the pixel values of the picturegeneration pixels in the picture data supplied from the image pickupdevice 200 (step S921). Then, 2D picture generation section 310interpolates the pixel values at the positions of the parallax detectionpixels in the generated planar picture (step S922).

Subsequently, the parallax detection section 320 acquires, from theposture detection section 140, the information (the posture information)related to the posture when the image pickup device 200 captures animage of the object (step S923). Then, the lateral direction (thehorizontal direction) is detected based on the posture information, andthe parallax detection section 320 determines the pixel circuitsgenerating left-eye data and the pixel circuits generating right-eyedata, out of the nine pixel circuits in the parallax detection pixel(step S924).

Subsequently, the parallax detection section 320 generates a left-eyeinformation picture based on the pixel values of the pixel circuitsgenerating the left-eye data out of the pixel values in the picture datasupplied from the image pickup device 200 (step S925). In addition, theparallax detection section 320 generates a right-eye information picturebased on the pixel values of the pixel circuits generating the right-eyedata out of the pixel values in the picture data supplied from the imagepickup device 200 (step S926). Then, the parallax detection section 320generates the parallax information picture based on the generatedleft-eye information picture and the generated right-eye informationpicture (step S927). Note that steps S925, S926, and S927 are an exampleof the procedure of detecting parallax described in CLAIMS. In addition,steps S921 and S922 are an example of the procedure of generating aplanar picture described in CLAIMS.

Subsequently, the 3D picture generation section 330 generates a left-eyepicture and a right-eye picture, based on the planar picture and theparallax information picture (step S928). Then, after step S928, thestereoscopic picture generation processing procedure is ended. Note thatstep S928 is an example of a stereoscopic picture generation proceduredescribed in CLAIMS.

As described above, according to the first embodiment of the technology,it is possible to generate the stereoscopic picture with high resolutionby moving the position of each object in the picture that is generatedby the picture generation pixels based on the parallax detected by theparallax detection pixels. In particular, according to the firstembodiment of the technology, it is possible to suppress decrease in thenumber of effective pixels in the image pickup unit that generates astereoscopic picture with use of the parallax detection pixels. Forexample, in an image pickup device in which all pixels in the imagepickup device are parallax detection pixels each configured of pixelcircuits of three rows and three columns, the number of effective pixelsis decreased to one-ninth. In contrast thereto, in the image pickupdevice 200 according to the first embodiment of the technology, sincethe parallax detection pixels are arranged in lines in the row directionand the column direction with predetermined intervals, it is possible tosuppress decrease in the number of effective pixels as the intervalbetween lines is increased. Moreover, since the picture generation pixelin which one pixel circuit configures one pixel is arranged in the imagepickup device 200, it is possible to capture the planar picture (the 2Dpicture), and accordingly, a stereoscopic picture or a planar picture isselectable according to the purpose of the user.

2. Second Embodiment

In the first embodiment of the technology, an example in which astereoscopic picture is generated based on the pixel values generated bythe nine pixel circuits configuring the parallax detection pixel isdescribed. Note that, since the pixel vales generated by the nine pixelcircuits in the parallax detection pixel include information related todefocusing, the pixel values may be used for focus determination by aphase difference detection system. In this case, phase differencedetection is a method of detecting a focal point in which light that haspassed through the image pickup lens is divided by pupil division toform a pair of images, and a distance between the formed images (adisplacement amount between the images) is measured (a phase differenceis detected) to detect degree of focusing.

Therefore, in a second embodiment of the technology, an example of animage pickup unit that performs focus determination by the phasedifference detection with use of parallax detection pixels in additionto generation of a stereoscopic picture will be described with referenceto FIG. 12 to FIG. 16.

[Example of Functional Configuration of Image Pickup Unit]

FIG. 12 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 400 according to the secondembodiment of the technology.

The image pickup unit 400 in the figure is provided with a focusdetermination section 410 in addition to the components of the imagepickup unit 100 illustrated in FIG. 1. The components other than thefocus determination section 410 in FIG. 12 are similar to the componentsillustrated in FIG. 1. Therefore, the description is given focusing onthe focus determination section 410.

The focus determination section 410 determines, based on the signal fromthe parallax detection pixel, whether focus is adjusted. The focusdetermination section 410 determines whether an object (an object to befocused) is focused in a region for focusing (a focus region), based onthe pixel values of the nine pixel circuits in the parallax detectionpixel in the captured picture data supplied from the image pickup device200. When it is determined that the object is focused, the focusdetermination section 410 supplies information indicating that focus isadjusted, as focus determination result information to the drive section170. In addition, when it is determined that the object to be focused isout of focus, the focus determination section 410 calculates an amountof defocusing (a defocusing amount), and supplies information indicatingthe calculated defocusing amount as the focus determination resultinformation to the drive section 170.

Incidentally, the drive section 170 calculates a drive amount of thefocus lens 113 based on the focus determination result informationoutput from the focus determination section 410, and moves the focuslens 113 according to the calculated drive amount. When focus has beenadjusted, the drive section 170 allows the focus lens 113 to bemaintained at the current position. Moreover, when focus has not beenadjusted, the drive section 170 calculates the drive amount (movementlength) based on the focus determination result information indicatingthe defocusing amount and the positional information of the focal lens113, and moves the focus lens 113 according to the drive amount.

[Concept Example of Automatic Focus using Parallax Detection Pixel]

FIG. 13 is a diagram schematically illustrating a concept of automaticfocus using the pixel values of the nine pixel circuits in the parallaxdetection pixel 230 according to the second embodiment of thetechnology.

Incidentally, in the figure, it is assumed that focusing is detected bygenerating a pair of images in the horizontal direction (in the lateraldirection in the figure) at the time when the picture is captured in thehorizontal shooting.

In (a) of the figure, a planar picture (a captured planar picture 440)of picture data captured by the image pickup device 200 for focusdetermination is illustrated. In the captured planar picture 440, anautomobile (an automobile 441), a person (a person 443), and a tree (atree 444) are shown as objects to be captured. In addition, in thecaptured planar picture 440, a chained rectangle indicating a focusregion (a focus region 442) is also shown. Incidentally, toschematically illustrate defocusing, the person 443 is enclosed bydouble dashed lines, and the automobile 441 is enclosed by fourfolddashed lines.

Note that, in the figure, the number of dashed lines enclosing theautomobile 441, the person 443, and the tree 444 indicates the degree ofdefocusing, and the focus is largely deviated as the number of dashedlines is increased. Specifically, in the figure, it is assumed that thetree 444 is focused at the time of capturing the captured planar picture440, and the person 443 and the automobile 441 are out of focus (theautomobile 441 is defocused more than the person 443).

(b) of the figure illustrates two pictures (a left-eye informationpicture 450 and a right-eye information picture 460) that areschematically illustrated as a pair of pictures corresponding to thecaptured planar picture 440 illustrated in (a) of the figure, for a pairof images generated by the focus determination section 410. Since theparallax detection pixels 230 are arranged in lines in the row directionand the column direction in the image pickup device 200 (see FIG. 2),the images shown in the left-eye information picture 450 and theright-eye information picture 460 are not generated actually. In FIG. 9,however, the description is made with schematic illustration of the pairof images in the entire captured picture (the captured planar picture440) for convenience of description.

The left-eye information picture 450 illustrates a picture generatedbased on the pixel values of the pixels corresponding to the left eyeout of the nine pixel circuits in the parallax detection pixel 230. Anautomobile 451 corresponding to the automobile 441 in (a) of FIG. 13, aperson 453 corresponding to the person 443 in (a) of the figure, and atree 454 corresponding to the tree 444 in (a) of the figure are shown inthe left-eye information picture 450.

Moreover, the right-eye information picture 460 illustrates a picturegenerated based on the pixel values of the pixels corresponding to theright eye out of the nine pixel circuits in the parallax detection pixel230. An automobile 461 corresponding to the automobile 441 in (a) ofFIG. 13, a person 463 corresponding to the person 443 in (a) of thefigure, and a tree 464 corresponding to the tree 444 in (a) of thefigure are shown in the right-eye information picture 460.

In (c) of the figure, a picture (a comparative picture 470)schematically illustrating measurement of a distance between the pair ofimages (displacement amount between the images) by the focusdetermination section 410. In the comparative picture 470, anillustrated image is like an image in which the left-eye informationpicture 450 and the right-eye information picture 460 illustrated in (b)of the figure are overlapped, and an automobile 471, a person 473, and atree 474 are shown. The automobile 471 is shown as an image in which adark automobile is present on the left of a pale automobile. Inaddition, the person 473 is shown as an image in which a dark person ispresent on the left of a pale person. On the other hand, the tree 474 isshown as an image in which a dark tree and a pale tree are coincidentwith each other.

As illustrated in the comparative picture 470, the focus determinationsection 410 compares the pair of images (in the figure, the left-eyeinformation picture 450 and the right-eye information picture 460).

Then, the focus determination section 410 calculates the amount ofdefocusing (the defocusing amount) from the displacement amount of thepair of images (a distance between edges) in the object to be focused(the person 473). The focus lens 113 is driven based on the defocusingamount so that the object to be focused is focused.

In (d) of the figure, the captured picture (a captured planar picture490 after a lens is driven) after the focus lens 113 is driven based onthe comparative picture 470 in (c) of the figure, and a picture (acomparative picture 480 after the lens is driven) indicating focusdetermination (image comparison) performed based on the shooting areillustrated.

In the comparative picture 480 after the lens is driven, a pictureschematically illustrating measurement of the distance between the pairof images after the lens is driven based on the comparative picture 470is illustrated. A tree 484 in which a dark tree is present on the rightof a pale tree, a person 483 in which a dark person is coincident with apale person, and an automobile 481 in which a dark automobile is presenton the left of a pale automobile are shown in the comparative picture470.

In the captured planar picture 490 after the lens is driven, anautomobile (an automobile 491) enclosed by a dashed line, a person (aperson 493) without a dashed line, a tree (a tree 494) enclosed byfourfold dashed lines, and a focus region 492 are shown. As described in(a) of the figure, the dashed line schematically indicates the degree ofdefocusing. In other words, in the captured planar picture 490 after thelens is driven, the person 493 that is an object to be focused isfocused.

As described above, in the image pickup unit 400, the focus lens 113 isdriven so that the pair of images for an object to be focused (person)is coincide with each other, and thus automatic focus is performed.

Note that, in FIG. 13, the concept of the automatic focus using thepixel values of the nine pixel circuits in the parallax detection pixel230 is described. Next, in FIG. 14, description is given focusing on acomparison of data at the time of focus determination performed by thefocus determination section 410.

[Example of Focus Determination by Focus Determination Section]

FIG. 14 is a diagram schematically illustrating focus determinationusing phase difference detection by the focus determination section 410according to the second embodiment of the technology.

Incidentally, in the figure, the focus determination by the focusdetermination section 410 in the case illustrated in FIG. 13 isdescribed. Note that, in the figure, for convenience of description, thedescription is given assuming that the number of lines of the parallaxdetection pixels 230 used for forming the pair of images is one.

In (a) of the figure, the pixel circuits generating signals related toone (right eye) and the other (left eye) of the pair of images areillustrated in the parallax detection pixel 230 in the case where thepair of images are formed in the lateral direction (the horizontaldirection) as illustrated in (b) of FIG. 13. In the parallax detectionpixel 230, the pixel circuits generating the signals related to the one(right eye) of the images are indicated by rectangles with a lot of dots(the pixel circuits 1, 4, and 7), and the pixel circuits generating thesignals related to the other (left eye) of the images are indicated byrectangles with small number of dots (the pixel circuits 3, 6, and 9).

In (b) of FIG. 14, the positions of the parallax detection pixels 230(focus determination line 421) whose pixel values are used in the focusdetermination section 410 (in which the pair of images are formed fromits pixel values) is illustrated in order to calculate defocusing amountwith respect to a set object to be focused. The pixel values of therespective pixel circuits in the parallax detection pixels 230 on thefocus determination line 421 are used as illustrated (a) of the figure,and therefore the pair of images is formed.

The determination of the direction (the lateral direction or verticaldirection) in which the phase difference is detected by the focusdetermination section 410 is briefly described now.

The focus determination section 410 determines a direction in which thedisplacement between the pair of images is detected, based on the pixelvalues related to the parallax detection pixel 230 of the picture datasupplied from the image pickup unit 200. Depending on the shape or thepattern of the object to be focused, there is a case where displacementof images is accurately detected by forming the pair of images in thelateral direction (the horizontal direction), and a case wheredisplacement of the images is accurately detected by forming the pair ofimages in the vertical direction (the gravitational-force direction).Therefore, the focus determination section 410 forms a pair of imagesbased on the pixel values of the parallax detection pixels 230 arrangedin line in a direction in which displacement of images is accuratelydetected.

In the figure, it is assumed that the direction in which thedisplacement amount is accurately detected is the lateral direction (thehorizontal direction). Therefore, the pair of images is formed based onthe pixel values of the parallax detection pixels 230 in the line inwhich the object to be focused is captured, out of the parallaxdetection pixel lines in the row direction. Note that the defocusingamount is allowed to be calculated if the displacement of edge of theobject to be focused (the person 422) is detected. Accordingly, the pairof images is formed from the pixel values of the parallax detectionpixels (on the focus determination line 421) located in the vicinity ofthe focus region.

In (c) of the figure, a graph schematically illustrating calculation ofthe defocusing amount by the focus determination section 410 isillustrated. In (c) of the figure, as illustrated in FIG. 13, thedescription is given assuming that the object to be focused is locatedon a side closer to the lens than the focus surface.

(c) of FIG. 14 illustrates a graph indicating the pair of imagesgenerated by the focus determination section 410 by distribution data ofpixel values, in the case where the horizontal axis indicates a pixelposition of the parallax detection pixel 230 in the image pickup device200, and the vertical axis indicates output gradation that indicatesintensity of an output signal. In the graph, pixel-value distributiondata (right-eye signal distribution data 431) generated based on thepixel values of the pixel circuits 1, 4, and 7 (corresponding to theright eye) illustrated in (a) of the figure is illustrated. In addition,in the graph, pixel-value distribution data (left-eye signaldistribution data 432) generated based on the pixel values of the pixelcircuits 3, 6, 9 (corresponding to the left eye) illustrated in (a) ofthe figure is illustrated.

The focus determination section 410 generates the right-eye signaldistribution data 431 and the left-eye signal distribution data 432 fromthe pixel values of the parallax detection pixels 230 in the focusdetermination line 421. Then, the focus determination section 410calculates the defocusing amount from a distance (a distance A1) betweena position of a peak (an edge) of the right-eye signal distribution data431 and a position of a peak (an edge) of the left-eye signaldistribution data 432. After that, the focus determination section 410supplies the calculated defocusing amount as the focus determinationresult information to the drive section 170, and drives the focus lens113 by an amount corresponding to the defocusing amount.

(d) of the figure illustrates the distribution data (right-eye signaldistribution data 433 and left-eye signal distribution data 434) in thecase where the focus lens 113 is driven based on the distribution data(the pair of images) illustrated in (c) of the figure and the object tobe focused is focused. As illustrated by the graph in (d) of the figure,when the object to be focused is focused, the position of the edge ofthe right-eye signal distribution data 433 is coincident with theposition of the edge of the left-eye signal distribution data 434.

As described above, in the image pickup unit 400, the focus lens 113 isdriven so that the positions of the edges of the pair of generatedimages (the distribution data) are coincident with each other asillustrated in (d) of the figure, and therefore, the automatic focus isperformed using the pixel values of the parallax detection pixels 230.In other words, in the image pickup unit 400, a focal point detectionmethod of phase difference detection system is achievable, and automaticfocus with high accuracy at high speed is accordingly achievable.

Incidentally, in the figure, although the description is given assumingthat the focus determination line is one line for convenience ofdescription, the number of lines is not limited thereto, and a pluralityof lines may be used to improve accuracy.

[Operation Example of Image Pickup Unit]

Next, operation of the image pickup unit 400 according to the secondembodiment of the technology is described with reference to figures.

FIG. 15 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 400 according to the second embodiment of thetechnology. Note that the flowchart of the example of the image pickupprocessing procedure illustrated in the figure is a modification of theflowchart of the example of the image pickup processing procedureaccording to the first embodiment of the technology illustrated in FIG.10. A difference point therebetween is the fact that the focusingprocessing (step S904) illustrated in FIG. 10 is changed to focusingprocessing (step S940). Therefore, the processing other than thefocusing processing (step S940) is illustrated with use of like numeralsand description thereof will be omitted, and the focusing processing(step S940) will be described with reference to FIG. 16.

FIG. 16 is a flowchart illustrating an example of a processing procedureof the focusing processing (step S940) in the image pickup processingprocedure according to the second embodiment of the technology.

First, an image of the object is captured by the image pickup device200, and a picture used for focus determination is captured (step S941).Subsequently, the focus determination section 410 determines thedirection (the row direction or the column direction) of lines of theparallax detection pixels that generates the pair of images, accordingto the object to be focused (step S942). Then, the position of theparallax detection pixels (for example, the focus determination line 421in FIG. 14) that is used for comparison between the pair of images isdetermined according to the position of the object to be focused (theposition of the focus region) (step S943).

Then, the pair of images is generated from the pixel values of the pixelcircuits in the parallax detection pixels at the determined positionwhere the pair of images is compared with each other (step S944). Afterthat, a distance between edges of the pair of generated images isdetected, and the defocusing amount is calculated from the distance(step S945). Subsequently, the drive section 170 calculates the driveamount of the focus lens 113 based on the calculated defocusing amount(step S945). Then, the focus lens 113 is driven based on the calculateddrive amount (step S947), and thus the focusing processing procedure isended.

As described above, according to the second embodiment of thetechnology, the focus determination by the phase difference detection isallowed to be performed based on the pixel values of the nine pixelcircuits configuring the parallax detection pixel. In other words,according to the second embodiment of the technology, it is possible torealize the image pickup unit that generates a 3D picture with the highnumber of pixels and performs automatic focus with high accuracy at highspeed by the phase difference detection.

3. Modifications

In the first and second embodiments of the technology, an example of theimage pickup device 200 in which the parallax detection pixels 230 arearranged in lines in the row direction and the column direction has beendescribed. However, the arrangement of the parallax detection pixels 230is not limited thereto, and any other arrangement is possible as long asthe parallax of the object is acquired. Therefore, examples of thearrangement of the parallax detection pixels different from thearrangement of the first and second embodiments will be described as afirst modification and a second modification with reference to FIG. 17and FIG. 18, respectively.

Moreover, in the first and second embodiments of the technology, asillustrated in (b) of FIG. 4, the case where the two microlens layersare provided and the microlens collecting light on each pixel circuit isformed in the microlens layer close to the color filter (close to thephotodetector) has been described. However, the technology is notlimited thereto, and various examples are conceivable in terms ofarrangement of the microlenses. For example, the case where themicrolens collecting light on each pixel circuit is formed in the samelayer as the layer of the parallax-detection-use microlens isconceivable. In addition, by setting a distance between theparallax-detection-use microlens and the nine photodetectors of theparallax detection pixel to be variable, the image pickup unit withinterchangeable image pickup lens (for example, a single-lens reflexcamera) may handle image pickup lenses with various f-numbers.Therefore, examples of an arrangement of microlenses different from thearrangement of the first and second embodiments will be described asthird to fifth modifications with reference to FIG. 19.

Moreover, in the first and second embodiments of the technology, theparallax detection pixel having the nine pixel circuits provided withthe G-filter has been described as an example. However, this is notlimitative. Examples of a parallax detection pixel different from theparallax detection pixel of the first and second embodiments will bedescribed as sixth to ninth modifications with reference to FIG. 20.

[Modifications of Arrangement of Pixels in Image Pickup Device]

FIG. 17 is a diagram schematically illustrating an example of an imagepickup device in which parallax detection pixels are arranged in linesonly in a row direction, as a first modification of the first and secondembodiments of the technology.

In (a) of the figure, a region (a region 810) corresponding to theregion 250 in (b) of FIG. 2 is illustrated. In other words, heavy lines(parallax detection pixel lines 811) in the region 810 indicate lines inwhich the parallax detection pixels are arranged. As illustrated by theparallax detection pixel lines 811, the parallax detection pixels arearranged in lines only in a row direction in an arbitrarily pixel cyclein the modification.

(b) of FIG. 17 is an enlarged view of a region 815 illustrated in (a) ofthe figure and illustrates the region corresponds to the region 210 in(a) of FIG. 2, and illustrates the fact that the parallax detectionpixels are arranged in lines only in the row direction.

In the image pickup unit (for example, a camcorder) that is rarely usedfor the vertical shooting, even if the parallax detection pixels arearranged in lines in a column direction, the parallax detection pixelsare hardly used for parallax detection. Therefore, the number ofparallax detection pixels are decreased to increase the number ofpicture generation pixels by arranging the line of the parallaxdetection pixels only in the row direction, and thus it is possible toimprove the quality of the captured picture, and to lighten theinterpolation processing of the pixel values at the positions of theparallax detection pixels.

FIG. 18 is a diagram schematically illustrating an example of an imagepickup device in which parallax detection pixels are arranged (arrangedin an island shape) in the row direction and the column direction withrespective predetermined intervals as the second modification of thetechnology.

In (a) of the figure, a region (a region 820) corresponding to theregion 250 in (b) of FIG. 2 is illustrated. In the region 820, a blackspot (a spot 821) indicates a position where one parallax detectionpixel is disposed. In other words, as illustrated by the spot 821, inthe modification, the parallax detection pixels are arranged (arrangedin an island shape) in the row direction and the column direction withpredetermined intervals.

(b) of FIG. 18 is an enlarged view of a region 825 illustrated in (a) ofthe figure, and illustrates a region corresponding to the region 210 in(a) of FIG. 2. The figure illustrates the fact that one parallaxdetection pixel is arranged in an island shape in the region 825.

As illustrated in FIG. 18, the number of parallax detection pixels isdecreased to be lower than that in the first modification to furtherincrease the number of picture generation pixels by arranging theparallax detection pixels in the row direction and the column directionwith predetermined intervals, and thus it is possible to improve thequality of the captured picture, and to lighten the interpolationprocessing of the pixel values at the positions of the parallaxdetection pixels.

As described above, various patterns of the arrangement of the parallaxdetection pixels in the image pickup device are conceivable.

[Modification of Cross-Sectional Structure of Picture Generation Pixeland Parallax Detection Pixel]

FIG. 19 is a diagram schematically illustrating a modification of across-sectional structure of the picture generation pixel and theparallax detection pixel as the third to fifth modifications of thetechnology.

(a) of the figure illustrates, as the third modification of thetechnology, an example of a cross-sectional structure of an image pickupdevice capable of varying a distance between a parallax-detection-usemicrolens and nine pixel circuits of a parallax detection pixel. Notethat (a) of the figure is a modification of the cross-sectionalstructure illustrated in (b) of FIG. 4, and is different in that amicrolens layer (a parallax-detection-use microlens arrangement section831) movable in an optical axis direction of the microlens is providedin place of the parallax-detection-use microlens layer S2 in (b) of FIG.4. Therefore, in (a) of FIG. 19, like numerals are used to designatelike components in (b) of FIG. 4 to omit the description thereof, anddescription is given of only the parallax-detection-use microlensarrangement section 831.

The parallax-detection-use microlens arrangement section 831 is amicrolens layer capable of varying the distance between theparallax-detection-use microlens 231 and the photodetectors of the pixelcircuits in the parallax detection pixel by moving in the optical axisdirection of the microlens. Specifically, an air layer (an air layer832) is present between the image pickup device and theparallax-detection-use microlens arrangement section 831. In this way,by allowing the distance between the parallax-detection-use microlens231 and the photodetectors of the pixel circuits in the parallaxdetection pixel to be variable, the image pickup unit may handleinterchangeable lenses with different f-numbers and different focallengths.

(b) of FIG. 19 illustrates an example of a cross-sectional structure ofan image pickup device in which a microlens collecting object light on aphotodetector of a picture generation pixel is provided also in aparallax-detection-use microlens layer, as the fourth modification ofthe technology. Configurations other than the microlens (a microlens833) provided in the parallax-detection-use microlens layer forcollecting the object light on the photodetector of the picturegeneration pixel are the same as those in (b) of FIG. 4, and thusdescription thereof is omitted in (b) of FIG. 19.

The microlens 833 is a microlens having a curvature different from thatof the parallax-detection-use microlens. Even in the case where imagingby the parallax detection pixel is prioritized and focus is adjusted,the object light is appropriately collected on the microlens 261 of thepicture generation pixel with the provided microlens 833. In otherwords, with the provided microlens 833, the microlens 261 appropriatelycollects the object light on the photodetector, and thus it is possibleto prevent deterioration of light collection efficiency of the picturegeneration pixel. Accordingly, it is possible to improve the picturequality.

(c) of FIG. 19 illustrates an example of a cross-sectional structure ofan image pickup device not including the microlens layer S1 of the imagepickup device in (b) of the figure, as the fifth modification of thetechnology. When parallax is detected only by the parallax-detection-usemicrolens 231 as well as the object light is collected on thephotodetector of the picture generation pixel only by the microlens 833,the microlens layer S1 may be omitted.

As described above, various patterns of the arrangement of themicrolenses in the image pickup device are conceivable.

[Example of Parallax Detection Pixel]

FIG. 20 is a schematic diagram illustrating modifications of theparallax detection pixel as sixth to ninth modifications of thetechnology.

(a) of the figure illustrates a parallax detection pixel (a parallaxdetection pixel 841) in which components allowing light within a visiblelight range to pass therethrough (for example, a transparent layer and aW filter) are provided in a color filter layer, as the sixthmodification of the technology.

(b) of the figure illustrates a parallax detection pixel (a parallaxdetection pixel 842) provided with the R filter, as the seventhmodification of the technology.

It is sufficient for the parallax detection pixel to detect parallax,and thus it is only necessary to set the filter of the parallaxdetection pixel according to the purpose. For example, as illustrated in(a) of the figure, parallax information less depending on color isobtainable by providing components that allow all light within thevisible light range to pass therethrough. In addition, when thecomponents allow all light within the visible light range to passtherethrough, a neutral density filter may be provided to adjustexposure because the light amount is possibly too much compared with thepicture generation pixel.

In addition, as illustrated in (b) of the figure, color filters otherthan the G filter may be provided in the parallax detection pixel.

(c) of the figure illustrates a parallax detection pixel (a parallaxdetection pixel 843) configured of a lot of pixel circuits, as theeighth modification of the technology. The parallax detection pixel 843is configured of pixel circuits of five rows and five columns (pixelcircuits 1 to 25). In addition, color filters of three primary colors (Rfilter, G filter, and B filter) are arranged based on Bayer arrangementin the pixel circuits 1 to 25.

As illustrated in (c) of the figure, one parallax detection pixel isconfigured of a lot of pixel circuits, and the color filters of threeprimary colors are arranged based on Bayer arrangement, and thereforeparallax information including color information is obtainable.

(d) of the figure illustrates a parallax detection pixel (a parallaxdetection pixel 844) configured of pixel circuits of two rows and twocolumns, as the ninth modification of the technology. In this way, theparallax detection pixel 844 may be configured of pixel circuits of tworows and two columns. In this case, compared with the case where theparallax detection pixel is configured of pixel circuits of three rowsand three columns, the number of pixel circuits used in one parallaxdetection pixel is allowed to be decreased, and the number of picturegeneration pixels and the number of parallax detection pixels areallowed to be increased. In other words, it is possible to improvepicture quality by increasing the number of picture generation pixels,and to improve accuracy of the parallax detection by increasing thenumber of parallax detection pixels.

4. Third Embodiment

In the first and second embodiments of the technology, the image pickupunit that generates a stereoscopic picture with use of the parallaxdetection pixels arranged in lines in the image pickup device has beendescribed. The image pickup unit described in the first and secondembodiments is a monocular image pickup unit.

In the monocular image pickup unit, stereoscopic effect of thestereoscopic picture depends on a distance (base-line length) between acentroid position of a region of the exit pupil where left-eye lightpasses and a centroid position of a region of the exit pupil whereright-eye light passes. The parallax amount increases as the base-linelength is longer, and as a result, the stereoscopic effect of thestereoscopic picture is increased.

The parallax detection pixel illustrated in the first and secondembodiments is configured of pixel circuits of three rows and threecolumns. When parallax in the lateral direction is detected, thedistance between the centroid position in the exit pupil of the objectlight received by the left pixel circuits and the centroid position inthe exit pupil of the object light received by the right pixel circuitsis a base-line length. In other words, to increase the base-line length,it is necessary to provide an image pickup lens with low f-number (largeexit pupil) or to provide an image pickup lens with a long focal length.The f-number is a parameter for setting brightness, and the focal lengthis a parameter for setting an angle of view. Therefore, these are notintended to be set freely only for the base-line length.

For example, at the time of shooting a bright scene, when the shootingis performed while the diaphragm is stopped down to increase thef-number (to decrease the size of the exit pupil), a favorable picturewithout halation is obtainable, whereas the base-line length isdecreased and stereoscopic effect is impaired. In addition, at the timeof shooting the bright scene, when the shooting is performed while thediaphragm is opened to decrease the f-number for the base-line length,the base-line length is increased and the stereoscopic effect isincreased, whereas halation occurs to deteriorate the picture quality.

In other words, it is desirable to provide a monocular image pickup unitthat achieves freely change both in base-line length and in brightness.Therefore, in the third embodiment of the technology, an example of amonocular image pickup unit that detects parallax in the horizontalshooting and is provided with a diaphragm capable of adjusting base-linelength and brightness is described with reference to FIG. 21 to FIG. 29.

[Example of Functional Configuration of Image Pickup Unit]

FIG. 21 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 500 according to a thirdembodiment of the technology.

The image pickup unit 500 illustrated in the figure includes a diaphragm510 in place of the diaphragm 112 of the image pickup unit 100illustrated in FIG. 1. In addition, the image pickup unit 500 includes abase-line length setting section 520 and a diaphragm drive settingsection 530 in addition to the components of the image pickup unit 100.Moreover, the image pickup unit 500 is not provided with the posturedetection section 140 of the image pickup unit 100. The diaphragm 510,the base-line length setting section 520, and the diaphragm drivesetting section 530 are described in FIG. 21.

Incidentally, it is assumed that the image pickup unit 500 is used onlyin the horizontal shooting. Therefore, it is assumed that the parallaxdetection pixels in the image pickup device 200 of the image pickup unit500 are arranged in lines only in the row direction of the image pickupdevice 200 as illustrated in (b) of FIG. 17. Further, it is assumed thateach of the parallax detection pixels in the image pickup device 200 ofthe image pickup unit 500 is configured of pixel circuits of two rowsand two columns as illustrated in (d) of FIG. 20. In other words, it isassumed that the parallax detection pixel is capable of dividing an exitpupil into a left part and a right part.

Moreover, the diaphragm 510 described in the third embodiment isapplicable to an image pickup unit that divides an exit pupil with useof any means (for example, a polarization device, a shutter, a mirror,optical vector information, or the like) into a left part and a rightpart to generate a stereoscopic picture. In other words, the thirdembodiment is not limited to the image pickup unit that generates astereoscopic picture with use of the parallax detection pixels describedin the first and second embodiments of the technology. In the thirdembodiment of the technology, however, description is given of thediaphragm 510 provided in the image pickup unit that generates astereoscopic picture with use of the parallax detection pixels.

The diaphragm 510 is a shielding component that varies degree of anaperture by drive of the drive section 170 and accordingly adjusts abase-line length and a light amount of object light entering the imagepickup device 200. The diaphragm 510 is capable of adjusting theposition of the aperture to set the base-line length, and is capable ofvarying the degree of the aperture at the set base-line length to adjustthe light amount of the object light. In other words, the diaphragm 510is capable of setting increase and decrease of the light amount of theobject light and the length of the base-line length independently. Notethat the diaphragm 510 will be described with reference to FIG. 22 toFIG. 25.

The base-line length setting section 520 sets the base-line lengthadjusted by the diaphragm 510. For example, the base-line length settingsection 520 calculates the base-line length according to the strength ofthe stereoscopic effect specified by a user through the operationreceiving section 120, and supplies information related to thecalculated base-line length (base-line length information) to thediaphragm drive setting section 530.

The diaphragm drive setting section 530 sets an aperture state of thediaphragm 510. For example, the diaphragm drive setting section 530calculates an appropriate light amount (automatic exposure (AE)) basedon picture data supplied from the image pickup device 200. Then, thediaphragm drive setting section 530 determines the aperture state of thediaphragm 510 based on the base-line length information supplied fromthe base-line setting section 520 and the calculated light amount. Inother words, the diaphragm drive setting section 530 sets the aperturestate of the diaphragm 510 to control increase and decrease of the lightamount of the object light and the length of the base-line lengthindependently. The diaphragm drive setting section 530 suppliesinformation related to the determined aperture state (diaphragm aperturestate information) to the drive section 170, and causes the drivesection 170 to drive the diaphragm 510.

[Example of Diaphragm]

FIG. 22 is a diagram schematically illustrating an example of thediaphragm 510 according to the third embodiment of the technology.

The diaphragm 510 includes a first diaphragm that is configured of twoblades forming an outer frame of an aperture of the diaphragm and asecond diaphragm that shields light vertically at the center of thediaphragm and in the vicinity thereof to generate right and leftapertures. In (a) of the figure, the diaphragm 510 in a state where twoapertures are formed by the first diaphragm and the second diaphragm isillustrated. In addition, in (b) of the figure, only the first diaphragm(a first diaphragm 511) is illustrated to illustrate the shape of twoblades of the first diaphragm. Then, in (c) of the figure, only thesecond diaphragm (a second diaphragm 515) is illustrated to illustratethe shape of two blades of the second diaphragm.

(a) of FIG. 22 illustrates the diaphragm 510 in a state where twoapertures are formed by four blades (a first diaphragm upper blade 512,a first diaphragm lower blade 513, a second diaphragm upper blade 516,and a second diaphragm lower blade 517) configuring the diaphragm 510.In addition, (a) of the figure illustrates centroid positions (centroidsP1 and P2) of the two apertures formed by the four blades and a distancebetween the two centroids (a base-line length L21).

(b) of the figure illustrates only the two blades (the first diaphragmupper blade 512 and the first diaphragm lower blade 513) configuring thefirst diaphragm (the first diaphragm 511). The first diaphragm upperblade 512 and the first diaphragm lower blade 513 are arranged so thattriangular concave cutouts face to each other. Note that the triangularconcave cutout is formed so that an apex of the triangular cutout islocated on a line that is a straight line orthogonal to the parallaxdirection (the lateral direction) and passes through the stop points ofthe base-line length. As illustrated in (b) of the figure, each of thepair of blades (the first diaphragm upper blade 512 and the firstdiaphragm lower blade 513) of the first diaphragm 511 is a planar lightshielding member so as to form an aperture of the diaphragm into asquare rotated by 45 degrees.

In addition, (c) of the figure illustrates only the two blades (thesecond diaphragm upper blade 516 and the second diaphragm lower blade517) configuring the second diaphragm (the second diaphragm 515). Asillustrated in (c) of the figure, each of the pair of blades (the seconddiaphragm upper blade 516 and the second diaphragm lower blade 517) ofthe second diaphragm 515 is a planar light shielding member thatprojects from top and bottom to shield light at the center of thediaphragm 510 and in the vicinity thereof. The second diaphragm upperblade 516 and the second diaphragm lower blade 517 are arranged so thattriangular (mountain-shaped) convex projections face to each other. Notethat the triangular convex projection is formed so that an apex of thetriangular cutout is located on a line that is a straight lineperpendicular (orthogonal) to the parallax direction and passes throughthe stop points of the base-line length. The pair of blades of thesecond diaphragm 515 has a shape in which the light-shielding areagradually increases from the center of the diaphragm 510 toward bothright and left ends while projecting. In the figure, as an example, asquare light shielding member rotated by 45 degrees is illustrated.

As illustrated in (a) to (c) of the figure, the diaphragm 510 isconfigured of the two blades (the first diaphragm upper blade 512 andthe first diaphragm lower blade 513) of the first diaphragm 511 and thetwo blades (the second diaphragm upper blade 516 and the seconddiaphragm lower blade 517) of the second diaphragm 515. Therefore, thediaphragm 510 forms a pair of apertures adjacent to each other in theparallax direction. In addition, the first diaphragm 511 forms half partof edge (a left half of the edge of the left aperture and a right halfof the edge of the right aperture) corresponding to both sides (bothends) in the parallax direction of both apertures, of the edges(peripheral edges) of the pair of apertures. The second diaphragm 515forms half part of edge (a right half of the edge of the left apertureand a left half of the edge of the right aperture) corresponding to theinside (the side where the pair of apertures is closed to each other) inthe parallax direction of both apertures, of the edges of the pair ofapertures. In other words, out of the peripheral edges of the pair ofapertures, the first diaphragm 511 forms end parts of the peripheraledges corresponding to both ends in the parallax direction, and thesecond diaphragm 515 forms closed parts of the peripheral edges that areclose to each other between the pair of apertures. Note that each of thecutouts of the first diaphragm 511 and the projections of the seconddiaphragm 515 is a triangle in which an apex thereof is located on aline that is a straight line perpendicular (orthogonal) to the parallaxdirection and passes through the stop points of the base-line length,and thus the shapes of the pair of the apertures are the same as eachother.

Next, change in the aperture shape caused by drive of the firstdiaphragm 511 and the second diaphragm 515 of the diaphragm 510 isdescribed with reference to FIG. 23 to FIG. 25.

[Example of Varying Aperture Area with Constant Base-Line Length]

FIG. 23 is a diagram schematically illustrating a driving direction ofthe first diaphragm 511 and the second diaphragm 515 when the diaphragm510 according to the third embodiment of the technology is driven sothat only an aperture area is varied with a constant base-line length.

(a) of the figure illustrates the diaphragm 510 before driving (referredto as a normal state).

(b) of the figure illustrates the diaphragm 510 that is driven so thatthe aperture area is decreased with the constant base-line length. Asillustrated in (b) of the figure, the drive of the diaphragm 510 fordecreasing the aperture area with the constant base-line length (thebase line length L31) is performed by topping down the first diaphragm(in a first diaphragm operation direction 551) and stopping down thesecond diaphragm (in a second diaphragm operation direction 552).

(c) of the figure illustrates the diaphragm 510 that is driven so thatthe aperture area is increased with the constant base-line length. Asillustrated in (c) of the figure, the drive of the diaphragm 510 forincreasing the aperture area with the constant base-line length (thebase-line length L31) is performed by opening the first diaphragm (in afirst diaphragm operation direction 553) and opening the seconddiaphragm (in a second diaphragm operation direction 554).

Specifically, as illustrated in (b) and (c) of the figure, when thelight amount is increased and decreased while the base-line length isfixed, according to the movement of the positions of the edges formed bythe first diaphragm in the parallax direction, the positions of theedges formed by the second diaphragm in the parallax direction are movedin a direction opposite to the moving direction of the positions of theedges of the first diaphragm, by the same amount as the moving amount ofthe positions of the edges of the first diaphragm. Therefore, a rightend of the left aperture and a left end of the left aperture are movedin opposite directions by the same amount, according to the movement ofthe left end of the left aperture, as well as a left end of the rightaperture and a right end of the right aperture are moved in oppositedirections by the same amount, according to the movement of the rightend of the right aperture. The setting of the edges of the apertures isachievable by driving the first diaphragm and the second diaphragm inthe same direction, as illustrated in (b) and (c) of the figure. In thisway, by setting the apertures of the diaphragm 510, it is possible toincrease and decrease the light amount without changing the centroidposition of each of the apertures.

[Example of Varying Base-Line Length with Constant Aperture Area]

FIG. 24 is a diagram schematically illustrating a driving direction ofthe first diaphragm 511 and the second diaphragm 515 when the diaphragm510 according to the third embodiment of the technology is driven sothat only the base-line length is varied with a constant aperture area.

(a) of the figure illustrates the diaphragm 510 in a normal state.

(b) of the figure illustrates the diaphragm 510 that is driven so thatthe base-line length is decreased from the normal state (from thebase-line length L31 to a base-line length L32) with the constantaperture area. As illustrated in (b) of the figure, the drive of thediaphragm 510 for decreasing the base-line length with the constantaperture area is performed by stopping down the first diaphragm (in afirst diaphragm operation direction 561) and opening the seconddiaphragm (in a second diaphragm operation direction 562).

(c) of the figure illustrates the diaphragm 510 that is driven so thatthe base-line length is increased from the normal state (from thebase-line length L31 to a base-line length L33) with the constantaperture area. As illustrated in (c) of the figure, the drive of thediaphragm 510 for increasing the base-line length with the constantaperture area is performed by opening the first diaphragm (in a firstdiaphragm operation direction 563) and stopping down the seconddiaphragm (in a second diaphragm operation direction 564).

Specifically, as illustrated in (b) and (c) of the figure, when thebase-line length is varied while the light amount is fixed, according tothe movement of the positions of the edges formed by the first diaphragmin the parallax direction, the positions of the edges formed by thesecond diaphragm in the parallax direction are moved in a direction sameas the moving direction of the positions of the edges of the firstdiaphragm, by the same amount as the moving amount of the positions ofthe edges of the first diaphragm. Therefore, the right end of the leftaperture and the left end of the left aperture are moved in the samedirection by the same amount, according to the movement of the left endof the left aperture, as well as the left end of the right aperture andthe right end of the right aperture are moved in the same direction bythe same amount, according to the movement of the right end of the rightaperture. The setting of the edges of the apertures is achievable bydriving the first diaphragm and the second diaphragm in oppositedirections, as illustrated in (b) and (c) of the figure. Accordingly, alength between the right end and the left end of the left aperture and alength between the right end and the left end of the right aperture areallowed to be fixed, and thus the aperture area of the aperture isallowed to be fixed. In this way, by setting the apertures of thediaphragm 510, it is possible to change the centroid position of each ofthe apertures without varying the aperture area.

[Example of Opening Second Diaphragm to Make One Aperture]

FIG. 25 is a diagram schematically illustrating a case where theaperture of the diaphragm 510 according to the third embodiment of thetechnology is formed into a shape suitable for capturing a planarpicture.

(a) of the figure illustrates the diaphragm 510 in the normal state.

(b) of the figure illustrates the diaphragm 510 that is driven so as toform the aperture into a shape suitable for capturing a planar picture.As illustrated in (b) of the figure, the diaphragm 510 has one aperturesimilarly to an existing diaphragm, by releasing the second diaphragm(in a second diaphragm driving direction 571).

Specifically, as illustrated in FIG. 23 and FIG. 24, in the diaphragm510, it is possible to set the aperture area (f-number) of the apertureand the base-line length (stereoscopic effect) independently by movingthe first diaphragm and the second diaphragm individually. For example,in shooting an extremely bright scene, to decrease the brightness withthe constant base-line length, the first diaphragm and the seconddiaphragm are both stopped down as illustrated in (b) of FIG. 23.Moreover, to enhance the stereoscopic effect while maintainingbrightness, the first diaphragm is opened and the second diaphragm isstopped down as illustrated in (c) of FIG. 24.

Further, as illustrated in FIG. 25, in capturing not a stereoscopicpicture but a planar picture, the second diaphragm is released to makeone aperture, and the aperture area is controlled by the firstdiaphragm. As a result, the diaphragm is allowed to be used like anexisting diaphragm.

As described above, the setting of the stereoscopic effect (3Dintensity) is achievable by the diaphragm 510. In addition, according tothe third embodiment of the technology in which the parallax detectionpixels and the picture generation pixels are arranged in the imagepickup device 200, a 2D picture (a planar picture) is allowed to begenerated based on the pixel values of the picture generation pixels. Inother words, providing the diaphragm 510 in the image pickup unit allowsa user to select a picture to be captured (2D picture or 3D picture) orto set stereoscopic effect.

Therefore, an example of a setting screen (a user interface) when a usersets a picture and 3D intensity is described with reference to FIG. 26.

[Example of Setting Screen in Display Section]

FIG. 26 is a diagram schematically illustrating a setting screen of apicture to be captured and a setting screen of 3D intensity that aredisplayed on the display section 151 according to the third embodimentof the technology.

In (a) of the figure, a setting screen (a setting screen 580) in which auser sets whether to capture a 2D picture or a 3D picture isillustrated. In the setting screen 580, a radio button (a radio button582) to select a 3D picture mode for capturing a 3D picture, and a radiobutton (a radio button 583) to select a 2D picture mode for capturing a2D picture are illustrated. Further, a determination button (ENTERbutton 584) to determine a selection and a button (BACK button 585) tocancel change of the selection are also illustrated.

In the setting screen 580, when the 3D picture mode is selected by theuser, the control of the diaphragm suitable for capturing a 3D pictureis performed as illustrated in FIG. 23 and FIG. 24. On the other hand,when the 2D picture mode is selected by the user, the second diaphragmis released and the control of the diaphragm suitable for capturing a 2Dpicture (a control similar to a control of an existing diaphragm) isperformed as illustrated in FIG. 25.

In this way, the image pickup unit 500 allows the user to select a 2Dpicture or a 3D picture as a picture to be captured.

(b) of FIG. 26 illustrates a setting screen (a setting screen 590) inwhich the user sets 3D intensity (3D level). In the setting screen 590,a slide bar (a slide bar 591) indicating the 3D level, a determinationbutton (ENTER button 594) for determining a selection, and a button(BACK button 595) for canceling the change of the selection areillustrated. In addition, in the slide bar 591, a bar (a bar 592)indicating the 3D level currently set.

In the setting screen 590, the user is allowed to select the 3D level bysliding the bar 592. When the 3D level is weakened (the bar 592 is movedtoward “weak” of the slide bar 591), the diaphragm 510 is controlled sothat the base-line length is decreased as illustrated in (b) of FIG. 24.On the other hand, the 3D level is increased (the bar 592 is movedtoward “strong” of the slide bar 591), the diaphragm 510 is controlledso that the base-line length is increased as illustrated in (c) of FIG.24.

As described above, the image pickup unit 500 allows the user to selectthe 3D level.

[Modification of 3D Level by Variation of Base-Line Length]

FIG. 27 is a diagram schematically illustrating change of an imagecaused by variation of the base-line length in the diaphragm 510according to the third embodiment of the technology.

(a) and (b) of the figure schematically illustrate an optical path froman object to be captured and an imaging position of the object to becaptured on an imaging surface when the diaphragm 510 is controlled sothat the base-line length is increased. In addition, (c) and (d) of thefigure schematically illustrate an optical path from the object to becaptured and an imaging position of the object to be captured on theimaging surface when the diaphragm 510 is controlled so that thebase-line length is decreased.

Note that (a) and (c) of the figure correspond to (a) of FIG. 6, andillustrate an optical path of object light that passes through anaperture plane (a left-eye aperture plane) corresponding to a left eyeof the diaphragm 510 of the object light from the object to be captured,and an imaging position on the imaging surface. Likewise, (b) and (d) ofFIG. 27 correspond to (b) of FIG. 6, and illustrate an optical path ofthe object light that passes through an aperture plane (a right-eyeaperture plane) corresponding to a right eye of the diaphragm 510 of theobject light from the object to be captured, and an imaging position onthe imaging surface. Therefore, the same components as those in (a) and(b) of FIG. 6 are attached with the same reference numerals as those ofFIG. 6, and description thereof will be omitted.

A pupil E21 illustrated in (a) to (d) of FIG. 27 schematicallyillustrates a shape of an exit pupil (that is, a shape of an imagepickup lens) when both the first diaphragm and the second diaphragm ofthe diaphragm 510 are released, and corresponds to the exit pupil E1 inFIG. 6. Exit pupils E31 and E32 in (a) and (b) of FIG. 27 illustrate apair of exit pupils (the exit pupil E31 illustrates an exit pupil of theleft-eye aperture plane, and the exit pupil E32 illustrates an exitpupil of the right-eye aperture plane) when the diaphragm 510 iscontrolled so that the base-line length is increased. Likewise, exitpupils E41 and E42 in (c) and (d) of the figure illustrate a pair ofexit pupils (the exit pupil E41 illustrates an exit pupil of theleft-eye aperture plane, and the exit pupil E42 illustrates an exitpupil of the right-eye aperture plane) when the diaphragm 510 iscontrolled so that the base-line length is decreased. In addition, in(a) to (d) of the figure, an optical path of light from the focusedobject 271 and an optical path of light from the close positioned object272 are indicated by dashed lines and solid lines (lines L51 to L58)extending from respective objects.

As illustrated in (a) and (b) of FIG. 27, when the base-line length isincreased (the left aperture plane and the right aperture plane are awayfrom each other), the displacement amount of the imaging positions thatare displaced according to the defocusing amount is increased, and thusstereoscopic effect is enhanced. On the other hand, as illustrated in(c) and (d) of the figure, when the base-line length is decreased (theleft aperture plane and the right aperture plane are close to eachother), the displacement amount of the imaging positions that aredisplaced according to the defocusing amount is decreased, and thus thestereoscopic effect is diminished.

As described above, providing the diaphragm 510 enables adjustment ofthe stereoscopic effect.

[Modification of Aperture Plane in Diaphragm]

FIG. 28 is a diagram schematically illustrating a difference between theaperture plane of the diaphragm 510 according to the third embodiment ofthe technology and an aperture plane of an existing diaphragm.

(a) of the figure illustrates change of the aperture plane by openingand closing a diaphragm (a diaphragm 190) provided in an existing imagepickup unit. The diaphragm 190 is configured of a pair of blades (adiaphragm upper blade 191 and a diaphragm lower blade 192), and an areaof the aperture plane is adjusted by moving the pair of blades (lightshielding members) in opposite directions. In the existing diaphragm190, the base-line length is long when the aperture area is large (see abase-line length L91 illustrated in (a) of the figure), whereas thebase-line length is short when the aperture area is small (see abase-line length L92 in (a) of the figure).

(b) of the figure illustrates change of the aperture by opening andclosing the diaphragm 510 according to the third embodiment of thetechnology. Note that a diagram illustrated in (b) of the figure is adiagram collecting FIG. 23 and FIG. 24. Therefore, in (b) of FIG. 28,the same reference numerals as those in FIG. 23 and FIG. 24 areattached, and detailed description is omitted here.

As illustrated in (b) of FIG. 28, the brightness (a size of the aperturearea) and the base-line length (the distance between the centroids ofthe pair of apertures) are allowed to be independently controlled by thediaphragm 510.

[Operation Example of Image Pickup Unit]

Next, operation of the image pickup unit 500 according to the thirdembodiment of the technology is described with reference to thedrawings.

FIG. 29 is a flowchart illustrating an example of an image pickupprocessing procedure when a stereoscopic picture is captured by theimage pickup unit 500 according to the third embodiment of thetechnology. Note that the flowchart of the example of the image pickupprocessing procedure illustrated in the figure is a modification of theflowchart of the example of the image pickup processing procedureaccording to the first embodiment of the technology illustrated in FIG.10. Therefore, like numerals are used to designate substantially likeprocesses of the first embodiment, and the description is given of anexample of an added processing procedure related to automatic exposure.

When it is determined that a starting instruction of image pickupoperation of a stereoscopic picture is issued by a user (step S901), thebase-line length is set based on the intensity of the stereoscopiceffect specified by the user in advance, and the diaphragm 510 is drivenaccording to the set base-line length (step S961). Then, the procedureproceeds to step S902, and display of live viewing is performed.

In addition, after the focusing processing is performed (step S904), thediaphragm drive setting section 530 adjusts exposure based on thepicture captured at the time of the focusing processing, and thusautomatic exposure processing in which the diaphragm 510 is controlledis performed (step S962). Then, after step S962, the procedure proceedsto step S905, and it is determined whether the shutter button is fullypushed or not.

As described above, according to the third embodiment of the technology,the brightness (the size of the aperture area) and the base-line length(the distance between the centroids of the pair of apertures) areallowed to be independently controlled. Note that, in the thirdembodiment of the technology, although the description is given on anassumption that the first diaphragm includes the two blades (lightshielding members) and the second diaphragm includes the two blades, theconfiguration is not limited thereto. Any other types of a firstdiaphragm may be used as long as the first diaphragm is opened andclosed so that an aperture area is decreased (the f-number is small)from an outer periphery of the diaphragm toward the center thereof.Increasing the number of blades changes the aperture shape from a squareshape rotated by 45 degrees to a circular shape. In addition, any othertypes of a second diaphragm may also be used as long as the seconddiaphragm shields light at the center of the diaphragm and in thevicinity thereof to form a pair of apertures, and it is conceivable thatusing two or more blades changes the shape of each of the pair ofapertures to a circular shape.

5. Fourth Embodiment

In the third embodiment of the technology, the diaphragm (the diaphragm510) in which the brightness and the base-line length are freely setwhen the horizontal shooting is performed by the image pickup unit hasbeen described. However, when the vertical shooting is performed asdescribed in the first embodiment of the technology, it is necessary torelease the second diaphragm and use only the first diaphragm (to usethe diaphragm in the same way as an existing diaphragm) in order toacquire parallax in the horizontal direction by the diaphragm 510.

Therefore, in a fourth embodiment of the technology, a diaphragm inwhich brightness and a base-line length are allowed to be freely setboth in the horizontal shooting and in the vertical shooting will bedescribed with reference to FIG. 30 to FIG. 32.

[Example of Functional Configuration of Image Pickup Unit]

FIG. 30 is a block diagram illustrating an example of a functionalconfiguration of an image pickup unit 600 according to the fourthembodiment of the technology.

The image pickup unit 600 illustrated in the figure includes a diaphragm610 in place of the diaphragm 510 of the image pickup unit 500illustrated in FIG. 21. Moreover, the image pickup unit 600 furtherincludes the posture detection section 140 illustrated in FIG. 1 inaddition to the components of the image pickup unit 500. Note that thediaphragm 610 will be described with reference to FIG. 31 and FIG. 32.

As with the posture detection section 140 illustrated in FIG. 1, theposture detection section 140 detects the posture of the image pickupunit 600, and supplies information (posture information) related to thedetected posture of the image pickup unit 600 to the parallax detectionsection 320 and the diaphragm drive setting section 530.

Note that the diaphragm drive setting section 530 illustrated in FIG. 30detects whether the shooting mode is the vertical shooting or thehorizontal shooting (the posture of the image pickup unit 600) based onthe posture information supplied from the posture detection section 140,and sets the drive of the diaphragm 610 according to the detectedposture.

[Example of Diaphragm]

FIG. 31 is a diagram schematically illustrating an example of thediaphragm 610 according to the fourth embodiment of the technology.

The diaphragm 610 includes, in addition to the first diaphragm and thesecond diaphragm that are the same as those in the diaphragm 510illustrated in FIG. 22, a third diaphragm that projects in the lateraldirection to shield light at the center of the diaphragm and in thevicinity thereof in order to form a pair of apertures at the top and thebottom. (a) of FIG. 31 illustrates the diaphragm 610 in a state wheretwo apertures are formed at the top and the bottom.

In addition, (b) of the figure illustrates only the first diaphragm (thefirst diaphragm 511) to illustrate the shape of the two blades of thefirst diaphragm, and (c) of the figure illustrates only the seconddiaphragm (the second diaphragm 515) to illustrate the shape of the twoblades of the second diaphragm. In addition, (d) of the figureillustrates only the third diaphragm (a third diaphragm 611) toillustrate the shape of the two blades of the third diaphragm.

Note that (b) and (c) of the figure are similar to (b) and (c) of FIG.22, and thus (a) and (d) of FIG. 31 are described.

(a) of the figure illustrates the diaphragm 610 in a state where thepair of apertures is formed in the vertical direction. In the diaphragm610, the third diaphragm is provided so that a right end of a thirddiaphragm left blade 612 and a left end of a third diaphragm right blade613 are in contact with each other in the vicinity of the center of thediaphragm 610. On the other hand, the second diaphragm upper blade 516and the second diaphragm lower blade 517 are in a released state, andare arranged so as not to shield the object light. In this way, light inthe vicinity of the center of the diaphragm 610 is shielded in thelateral direction by the third diaphragm 611, and thus the pair ofapertures is allowed to be formed in the vertical direction of thediaphragm 610.

(d) of the figure illustrates the two blades (the third diaphragm leftblade 612 and the third diaphragm right blade 613) configuring the thirddiaphragm (the third diaphragm 611). As illustrated in (d) of thefigure, the arrangement direction of the third diaphragm 611 correspondsto the arrangement direction obtained by rotating the arrangementdirection of the second diaphragm 515 (see (c) of the figure) by 90degrees in a clockwise direction (the drive directions thereof areorthogonal to each other). In other words, the third diaphragm leftblade 612 and the third diaphragm right blade 613 are arranged so thatthe triangular convex projections thereof face to each other. Note thatthe triangular convex projection is formed so that an apex of atriangular cutout is located on a line that is a straight line parallelto the parallax direction and passes through the stop points of thebase-line length.

As described above, providing the third diaphragm 611 that shields lightat the center of the diaphragm 610 and in the vicinity thereof in thelateral direction forms the pair of apertures in the vertical direction.

[Modification of Aperture by Diaphragm]

FIG. 32 is a diagram schematically illustrating an example of a shape ofthe apertures formed by the diaphragm 610 according to the fourthembodiment of the technology.

(a) of the figure schematically illustrates a position of each blade ofthe diaphragm 610 in the case where the horizontal shooting is performedusing the image pickup unit 600. In the case of performing thehorizontal shooting, the third diaphragm (the third diaphragm left blade612 and the third diaphragm right blade 613) is put into a releasedstate as illustrated in (a) of the figure. In addition, as illustratedin FIG. 22 to FIG. 24, the second diaphragm (the second diaphragm upperblade 516 and the second diaphragm lower blade 517) is stopped down(closed) so as to form the pair of apertures. Then, by opening andclosing the second diaphragm and the first diaphragm while remaining thethird diaphragm in the released state, the brightness and the base-linelength are allowed to be separately controlled in the horizontalshooting similarly to the diaphragm 510 illustrated in FIG. 22 to FIG.24.

(b) of FIG. 32 schematically illustrates a position of each blade of thediaphragm 610 in the case where the vertical shooting is performed usingthe image pickup unit 600. In the case of performing the verticalshooting, the second diaphragm is put into a released state and thethird diaphragm is stopped down (closed) so as to form the pair ofapertures as illustrated in (b) of the figure. Then, by opening andclosing the third diaphragm and the first diaphragm while remaining thesecond diaphragm in the released state, the opening and closing of thediaphragm are performed as with (a) of the figure except that thedirection of the pair of apertures is different. In other words, asillustrated in (b) of the figure, by opening and closing the thirddiaphragm and the first diaphragm while remaining the second diaphragmin the released state, the brightness and the base-line length areallowed to be separately controlled in the vertical shooting.

(c) of the figure schematically illustrates a position of each blade ofthe diaphragm 610 in the case where a 2D picture is captured using theimage pickup unit 600. In the case of capturing a 2D picture, the seconddiaphragm and the third diaphragm are put into a released state asillustrated in (c) of the figure. In addition, only the first diaphragmis opened and closed. As a result, a 2D picture is allowed to becaptured without unnecessarily shielding object light.

As described above, according to the fourth embodiment of thetechnology, the brightness and the base-line length are allowed to beindependently controlled both in the horizontal shooting and in thevertical shooting.

6. Modification of Diaphragm

In the third and the fourth embodiments of the technology, the diaphragmin which the pair of apertures is formed and the brightness and thebase-line length are allowed to be freely set has been described. Notethat, in the diaphragm described in the third and fourth embodiments,the brightness and the base-line length are allowed to be freelycombined. However, it may not necessary for a simple 3D image pickupunit to have such performance. In this case, a diaphragm having asimpler configuration suitable for capturing a 3D picture is demanded.

Therefore, a diaphragm having a configuration simpler than that in thethird and fourth embodiments is described with reference to FIG. 33.

[Example of Diaphragm]

FIG. 33 is a diagram schematically illustrating examples of a diaphragmhaving a simple configuration suitable for capturing a 3D picture asmodifications of the third and fourth embodiments of the technology.

(a) and (b) of the figure each illustrate a diaphragm in whichbrightness is allowed to be controlled while a base-line length ismaintained. (a) of the figure illustrates a diaphragm in which twoblades are provided and a rectangular aperture plane that is long in aparallax (lateral) direction is formed by projections at both ends (arectangular cutout is formed inside) in the parallax detectiondirection. The rectangular aperture plane of the diaphragm is formed bythe two blades in which rectangular cutouts each having a long side inthe parallax direction face to each other. In addition, (b) of thefigure illustrates a diaphragm in which two blades are provided and apair of apertures (squares rotated by 45 degrees) is formed in theparallax (lateral) direction by notch-cut projections (having twovalleys (a pair of adjacent cutouts)). By opening and closing thediaphragm illustrated in (a) and (b) of the figure in the verticaldirection (in a direction perpendicular to the parallax detectiondirection), the brightness is allowed to be controlled while maintainingthe base-line length.

(c) and (d) of the figure each illustrate a diaphragm in which abase-line length is allowed to be longer than that of an existingdiaphragm. (c) of the figure illustrates a diaphragm forming anelliptical aperture that is long in the lateral direction (in theparallax detection direction) and is short in the vertical direction.The elliptical aperture of the diaphragm is formed by two blades inwhich semicircular cutouts each having one side (long diameter) in theparallax direction face to each other. In addition, (d) of the figureillustrates a diaphragm forming a rhomboid aperture that is long in thelateral direction (the parallax detection direction) and is short in thevertical direction. The rhomboid aperture of the diaphragm is formed bytwo blades in which triangular cutouts each having a base in theparallax direction face to each other. By opening and closing thediaphragm illustrated in (c) and (d) of the figure in the verticaldirection, the base-line length is allowed to be increased, comparedwith an existing diaphragm having a circular aperture or a diaphragmhaving a square aperture rotated by 45 degrees.

(e) of the figure illustrates a diaphragm in which brightness and abase-line length are independently controlled as with the third andfourth embodiment of the technology. The diaphragm is controlled moreeasily than that of the third and fourth embodiments. In (e) of thefigure, two blades closing from both ends (a left end and a right end)in the parallax detection (lateral) direction toward the center of thediaphragm, and two blades closing from both ends (an upper end and alower end) in the vertical direction toward the center of the diaphragmare illustrated. Note that the two blades closing from the left end andthe right end toward the center of the diaphragm are paired to allowrespective sides parallel to the orthogonal direction orthogonal to theparallax direction to face each other. In addition, the two bladesclosing from the upper end and the lower end toward the center of thediaphragm are paired to allow respective sides parallel to the parallaxdirection to face each other. In the diaphragm illustrated in (e) of thefigure, the base-line length is increased when the right and leftdiaphragms are opened, and the brightness is increased when the upperand lower diaphragms are opened.

Providing such a diaphragm suitable for capturing a 3D picture in animage pickup unit enables capturing of a 3D picture providing favorablestereoscopic effects.

As described above, according to the embodiments of the technology, afavorable 3D picture is allowed to be captured. Incidentally, in theembodiments of the technology, although the description is made on anassumption that an image pickup unit captures a still picture, thetechnology is also embodied in an image pickup unit capturing a movingpicture similarly.

Note that the above-described embodiments are to illustrate examples forembodying the technology, and subject matter in the embodiments andinvention-specifying subject matter in claims have correspondencerelationship. Similarly, invention-specifying subject matter in claimsand subject matter attached with the same name as theinvention-specifying subject matter in the embodiments of the technologyhave correspondence relationship. Incidentally, the technology is notlimited to the embodiments, and is embodied by giving variousmodifications to the embodiments without departing from the scope of thetechnology.

In addition, the processing procedures described in the above-describedembodiments may be considered as a method having the series ofprocedures, and may be considered as program that causes a computer toexecute the series of procedures or as a recording medium holding theprogram. As the recording medium, for example, a compact disc (CD), amini disc (MD), a digital versatile disk (DVD), a memory card, Blu-raydisc (registered trademark) may be used.

Note that the technology may be configured as follows.

(1) An image pickup unit including:

-   -   an image pickup device including parallax detection pixels and        picture generation pixels, each of the parallax detection pixels        receiving object light by a plurality of photodetectors covered        by one microlens, to generate a signal used for detecting        parallax, and each of the picture generation pixels receiving        the object light by a photodetector covered, on a pixel basis,        with a microlens smaller in size than the microlens, to generate        a signal used for generating a picture; and    -   a stereoscopic picture generation section detecting the parallax        based on the signal generated by the parallax detection pixels,        generating a planar picture based on the signal generated by the        picture generation pixels, and adjusting a position of each        object image included in the generated planar picture, based on        the detected parallax, to generate a stereoscopic picture.

(2) The image pickup unit according to (1), further including a posturedetection section detecting a posture of the image pickup unit, wherein

-   -   the parallax detection pixels are arranged on a line in a row        direction of the image pickup device and on a line in a column        direction thereof, and    -   the stereoscopic picture generation section determines a        detection direction of the parallax from the row direction and        the column direction of the image pickup device, based on the        posture detected by the posture detection section, and generates        information related to the parallax, based on the signal        generated by the parallax detection pixels arranged in the        determined direction.

(3) The image pickup unit according to (1) or (2), further including afocus determination section performing focus determination on an objectto be focused, based on the signal generated by the parallax detectionpixels.

(4) The image pickup unit according to any one of (1) to (3), whereinthe parallax detection pixels in the image pickup device are arranged tobe adjacent to one another on a line in a specific direction.

(5) The image pickup unit according to any one of (1) to (3), whereinthe parallax detection pixels in the image pickup device are arrangedwith predetermined intervals on a line in a specific direction.

(6) The image pickup unit according to any one of (1) to (5), furtherincluding a control section that moves the one microlens covering theplurality of photodetectors in the parallax detection pixel, in anoptical axis direction of the microlens, based on a relationship betweenthe image pickup device and a size of the exit pupil.

(7) The image pickup unit according to any one of (1) to (6), whereinthe plurality of photodetectors in the parallax detection pixel arecovered with one color filter.

(8) The image pickup unit according to (7), wherein the plurality ofphotodetectors in the parallax detection pixel are covered with a greenfilter, the green filter shielding light other than light within awavelength range showing green.

(9) The image pickup unit according to (7), wherein the plurality ofphotodetectors in the parallax detection pixel are covered with a whitefilter or a transparent layer, the white filter and the transparentlayer allowing light within a visible light range to pass therethrough.

(10) The image pickup unit according to any one of (1) to (9), whereinthe picture generation pixel includes one photodetector on the pixelbasis.

(11) The image pickup unit according to any one of (1) to (10), whereina microlens that is for collecting the object light on a position ofeach of the plurality of photodetectors, covers

the plurality of photodetectors on the plurality of photodetectorsbasis, the object light being collected by the one microlens coveringthe plurality of photodetectors in the parallax detection pixel.

(12) The image pickup unit according to (11), wherein the microlenseseach covering the photodetector in the picture generation pixel arearranged on one surface orthogonal to an optical axis direction of themicrolens covering the plurality of photodetectors in the parallaxdetection pixel on the plurality of photodetectors basis.

(13) The image pickup unit according to any one of (1) to (11), whereinthe microlenses each covering the photodetector in the picturegeneration pixel are arranged on one surface orthogonal to an opticalaxis direction of the one microlens covering the plurality ofphotodetectors in the parallax detection pixel.

(14) An image pickup device including:

-   -   parallax detection pixels each receiving object light by a        plurality of photodetectors covered by one microlens, to        generate a signal used for detecting parallax, the parallax        being used for generating a stereoscopic picture; and    -   picture generation pixels each receiving the object light by a        photodetector covered, on a pixel basis, with a microlens        smaller in size than the microlens, to generate a signal used        for generating a planar picture, the planar picture being used        for generating the stereoscopic picture using the parallax.

(15) The image pickup device according to (14), wherein

-   -   the parallax is information related to a displacement amount of        a position of each object image in the planar picture at a time        when the position of each object image is adjusted in the        parallax direction to generate the stereoscopic picture, and    -   the parallax detection pixels are arranged on a line in the        parallax direction.

(16) A picture processing method including:

-   -   a step of detecting parallax based on a signal generated by        parallax detection pixels in an image pickup device, the image        pickup device including the parallax detection pixels and        picture generation pixels, the parallax detection pixels        generating a signal used for detecting parallax by a plurality        of photodetectors covered by one microlens, the picture        generation pixels each receiving object light to generate a        signal used for generating a planar picture;    -   a step of generating the planar picture based on the signal        generated by the picture generation pixels in the image pickup        device; and    -   a step of adjusting a position of each of captured objects in        the planar picture, based on the detected parallax, to generate        a stereoscopic picture.

(17) A program causing a computer to execute:

-   -   a step of detecting parallax based on a signal generated by        parallax detection pixels in an image pickup device, the image        pickup device including the parallax detection pixels and        picture generation pixels, the parallax detection pixels        generating a signal used for detecting parallax by a plurality        of photodetectors covered by one microlens, the picture        generation pixels each receiving object light to generate a        signal used for generating a planar picture;    -   a step of generating the planar picture based on the signal        generated by the picture generation pixels in the image pickup        device; and    -   a step of adjusting a position of each of captured objects in        the planar picture, based on the detected parallax, to generate        a stereoscopic picture.

(18) An image pickup unit including:

-   -   a diaphragm forming a pair of apertures for generating a        stereoscopic picture;    -   an image pickup device receiving object light that passes        through each of the pair of apertures, to generate signals used        for generating the stereoscopic picture; and    -   a control section independently controlling a distance between        centroids of the pair of apertures, and increase and decrease of        light amount of the object light passing through the pair of        apertures.

(19) The image pickup unit according to (18), wherein

-   -   the pair of apertures is formed, in the diaphragm, to be        adjacent to each other in a parallax direction of the        stereoscopic picture, and    -   the control section changes and controls, of peripheral edges of        the pair of apertures, positions of end parts of the peripheral        edges corresponding to both ends in the parallax direction and        positions of closed parts of the peripheral edges that are close        to each other between the pair of apertures.

(20) The image pickup unit according to (19), wherein, when the lightamount is increased or decreased, the control section varies a lengthbetween the end part of the peripheral edge corresponding to the end ofone of the pair of apertures and the closed part of the peripheral edgethereof, and a length between the end part of the peripheral edgecorresponding to the end of the other of the pair of apertures and theclosed part of the peripheral edge thereof, in a state where thedistance between the centroids is fixed.

(21) The image pickup unit according to (20), wherein the length betweenthe end part of the peripheral edge corresponding to the end of the oneof the pair of apertures and the closed part of the peripheral edgethereof is equal to the length between the end part of the peripheraledge corresponding to the end of the other of the pair of apertures andthe closed part of the peripheral edge thereof.

(22) The image pickup unit according to any one of (19) to (21),wherein, when the distance between the centroids is varied, the controlsection varies the distance between the centroids in a state where thelength between the end part of the peripheral edge corresponding to theend of the one of the pair of apertures and the closed part of theperipheral edge thereof is fixed.

(23) The image pickup unit according to any one of (19) to (22), furtherincluding an adjusting section adjusting the distance between thecentroids, wherein

-   -   the control section controls the pair of apertures to allow the        distance between the centroids to be a distance adjusted by the        adjusting section.

(24) The image pickup unit according to any one of (18) to (23), whereinthe diaphragm includes a first member that includes a pair of memberseach having a cutout and a second member that includes a pair of memberseach having a projection, the pair of members of the first member beingdisposed to allow the cutouts to face each other, and the pair ofmembers of the second member being disposed to allow the projections toface each other.

(25) The image pickup unit according to (24), wherein the first memberand the second member are driven in an orthogonal direction orthogonalto the parallax direction.

(26) The image pickup unit according to (25), wherein

-   -   the cutout has a concave shape in which an apex of a mountain        shape is located on a straight line that passes through a        midpoint of the distance between the centroids and is parallel        to a driving direction of the first member, and    -   the projection has a convex shape in which an apex of a mountain        shape is located on a straight line that passes through the        midpoint of the distance between the centroids and is parallel        to a driving direction of the second member.

(27) The image pickup unit according to any one of (18) to (23), furtherincluding a posture detection section detecting a posture of the imagepickup unit, wherein

-   -   the diaphragm includes a first member, a second member shielding        part of the object light in a horizontal shooting, and a third        member shielding part of the object light in a vertical        shooting, the first member having a pair of members each having        a cutout, the pair of members of the first member being disposed        to allow the cutouts to face each other, the second member        having a pair of members each having a projection, the pair of        members of the second member being disposed to allow the        projections to face each other, the third member having a pair        of members each having a projection, and the pair of the members        of the third member being disposed to allow the projections to        face each other,    -   a driving direction of the second member is orthogonal to a        driving direction of the third member, and    -   the control section determines, based on the detected posture,        whether the horizontal shooting or the vertical shooting is        performed, and then controls the pair of apertures.

(28) The image pickup unit according to any one of (18) to (27), whereinthe diaphragm is disposed on an optical path of the object light that iscollected by a monocular lens system.

(29) An image pickup unit including:

-   -   a diaphragm configured of a pair of members each having a pair        of cutouts that are adjacent to each other in a parallax        direction of a stereoscopic picture, the cutouts of the        respective members facing to each other to form a pair of        apertures;    -   an image pickup device receiving object light that passes        through each of the pair of apertures to generate a signal used        for generating the stereoscopic picture; and    -   a control section driving each of the pair of members in an        orthogonal direction orthogonal to the parallax direction and        controlling the diaphragm to allow a distance between centroids        of the pair of apertures to be fixed.

(30) An image pickup unit including:

-   -   a diaphragm forming an aperture, a longitudinal direction of the        aperture being a parallax direction in a stereoscopic picture;    -   an image pickup device receiving object light that passes        through the aperture, to generate a signal used for generating        the stereoscopic picture; and    -   a control section controlling the diaphragm to allow a length of        the aperture in the parallax direction to be larger than a        length of the aperture in an orthogonal direction orthogonal to        the parallax direction.

(31) The image pickup unit according to (30), wherein

-   -   the diaphragm includes a pair of members each having a cutout,        the cutouts facing to each other to form the aperture, and    -   the control section drives each of the pair of members in the        orthogonal direction to control the diaphragm.

(32) The image pickup unit according to (30), wherein the cutout has oneof a rectangular shape having a long side extending in the parallaxdirection, a triangular shape having a base extending in the parallaxdirection, and a semicircular shape having one side extending in theparallax direction.

(33) The image pickup unit according to (30), wherein the diaphragmforms the aperture with use of a pair of first members and a pair ofsecond members, the first members each having a side parallel to theparallax direction, the sides of the respective first members facing toeach other, the second members each having a side parallel to theorthogonal direction, and the sides of the respective second membersfacing to each other.

(34) A diaphragm control method including:

-   -   a first control step of controlling a distance between centroids        of a pair of apertures in a diaphragm, the diaphragm forming the        pair of apertures used for generating a stereoscopic picture;        and    -   a second control step of controlling increase and decrease of a        light amount of the object light that passes through the pair of        apertures, independently of the distance between the centroids.

(35) A program causing a computer to execute:

-   -   a first control step of controlling a distance between centroids        of a pair of apertures in a diaphragm, the diaphragm forming the        pair of apertures used for generating a stereoscopic picture;        and    -   a second control step of controlling increase and decrease of a        light amount of the object light that passes through the pair of        apertures, independently of the distance between the centroids.

REFERENCE SIGNS LIST

-   100 image pickup unit-   110 lens section-   111 zoom lens-   112 diaphragm-   113 focus lens-   120 operation receiving section-   130 control section-   140 posture detection section-   151 display section-   152 memory section-   170 drive section-   200 image pickup device-   230 parallax detection pixel-   300 signal processing section-   310 2D picture generation section-   320 parallax detection section-   330 3D picture generation section-   400 image pickup unit-   410 focus determination section-   500 image pickup unit-   510 diaphragm-   520 base-line length setting section-   530 diaphragm drive setting section

The invention claimed is:
 1. An image pickup unit comprising: adiaphragm configured to form a pair of apertures for generating astereoscopic picture, wherein the diaphragm includes a first member thatincludes a first blade and a second blade each having a cutout, and asecond member that includes a third blade and a fourth blade each havinga projection, the first blade and the second blade being disposed toallow the cutouts to face each other, and the third blade and the fourthblade being disposed to allow the projections to face each other,wherein the first member and the second member are driven in anorthogonal direction orthogonal to a parallax direction; an image pickupdevice configured to receive object light that passes through each ofthe pair of apertures, to generate signals used for generating thestereoscopic picture; and a control section configured to control adistance between centroids of the pair of apertures, and an increase anddecrease of a light amount of the object light passing through the pairof apertures, wherein the cutout has a concave shape in which an apex ofa mountain shape is located on a straight line that passes through amidpoint of the distance between the centroids and is parallel to adriving direction of the first member, the projection has a convex shapein which an apex of a mountain shape is located on a straight line thatpasses through the midpoint of the distance between the centroids and isparallel to a driving direction of the second member, and the distancebetween the centroids and the increase and decrease of the light amountare controlled independently of one another.
 2. The image pickup unitaccording to claim 1, wherein the pair of apertures is formed, in thediaphragm, to be adjacent to each other in a parallax direction of thestereoscopic picture, and the control section is configured to changeand control, of peripheral edges of the pair of apertures, positions ofend parts of the peripheral edges corresponding to both ends in theparallax direction and positions of closed parts of the peripheral edgesthat are close to each other between the pair of apertures.
 3. The imagepickup unit according to claim 2, wherein, when the light amount isincreased or decreased, the control section varies a length between theend part of the peripheral edge corresponding to the end of one of thepair of apertures and the closed part of the peripheral edge thereof,and a length between the end part of the peripheral edge correspondingto the end of the other of the pair of apertures and the closed part ofthe peripheral edge thereof, in a state where the distance between thecentroids is fixed.
 4. The image pickup unit according to claim 3,wherein the length between the end part of the peripheral edgecorresponding to the end of the one of the pair of apertures and theclosed part of the peripheral edge thereof is equal to the lengthbetween the end part of the peripheral edge corresponding to the end ofthe other of the pair of apertures and the closed part of the peripheraledge thereof.
 5. The image pickup unit according to claim 2, wherein,when the distance between the centroids is varied, the control sectionvaries the distance between the centroids in a state where the lengthbetween the end part of the peripheral edge corresponding to the end ofthe one of the pair of apertures and the closed part of the peripheraledge thereof is fixed.
 6. The image pickup unit according to claim 2,further comprising an adjusting section configured to adjust thedistance between the centroids, wherein the control section isconfigured to control the pair of apertures to allow the distancebetween the centroids to be a distance adjusted by the adjustingsection.
 7. An image pickup unit, comprising: a diaphragm configured toform a pair of apertures for generating a stereoscopic picture, whereinthe diaphragm includes a first member that includes a first blade and asecond blade each having a cutout, and a second member that includes athird blade and a fourth blade each having a projection, the first bladeand the second blade being disposed to allow the cutouts to face eachother, and the third blade and the fourth blade being disposed to allowthe projections to face each other, wherein the first member and thesecond member are driven in an orthogonal direction orthogonal to aparallax direction; an image pickup device configured to receive objectlight that passes through each of the pair of apertures, to generatesignals used for generating the stereoscopic picture; and a controlsection configured to control a distance between centroids of the pairof apertures, and an increase and decrease of a light amount of theobject light passing through the pair of apertures, wherein the cutouthas a concave shape in which an apex of a mountain shape is located on astraight line that passes through a midpoint of the distance between thecentroids and is parallel to a driving direction of the first member,the projection has a convex shape in which an apex of a mountain shapeis located on a straight line that passes through the midpoint of thedistance between the centroids and is parallel to a driving direction ofthe second member, and the distance between the centroids and theincrease and decrease of the light amount are controlled independentlyof one another; a posture detection section configured to detect aposture of the image pickup unit, wherein the diaphragm includes a firstmember, a second member shielding part of the object light in ahorizontal shooting, and a third member shielding part of the objectlight in a vertical shooting, the first member having a first blade anda second blade each having a cutout, the first blade and the secondblade being disposed to allow the cutouts to face each other, the secondmember having a third blade and a fourth blade each having a projection,the third blade and the fourth blade being disposed to allow theprojections to face each other, the third member having a fifth bladeand a sixth blade each having a projection, and the fifth blade and thesixth blade being disposed to allow the projections to face each other,a driving direction of the second member is orthogonal to a drivingdirection of the third member, and the control section is configured todetermine, based on the detected posture, whether the horizontalshooting or the vertical shooting is performed, and then to control thepair of apertures.
 8. The image pickup unit according to claim 1,wherein the diaphragm is disposed on an optical path of the object lightthat is collected by a monocular lens system.
 9. An image pickup unitcomprising: a diaphragm forming an aperture, a longitudinal direction ofthe aperture being a parallax direction in a stereoscopic picture; animage pickup device receiving object light that passes through theaperture, to generate a signal used for generating the stereoscopicpicture; and a control section controlling the diaphragm to allow alength of the aperture in the parallax direction to be larger than alength of the aperture in an orthogonal direction orthogonal to theparallax direction.
 10. The image pickup unit according to claim 9,wherein the diaphragm includes a pair of members each having a cutout,the cutouts facing to each other to form the aperture, and the controlsection drives each of the pair of members in the orthogonal directionto control the diaphragm.
 11. The image pickup unit according to claim10, wherein the cutout has one of a rectangular shape having a long sideextending in the parallax direction, a triangular shape having a baseextending in the parallax direction, and a semicircular shape having oneside extending in the parallax direction.
 12. The image pickup unitaccording to claim 9, wherein the diaphragm forms the aperture with useof a pair of first members and a pair of second members, the firstmembers each having a side parallel to the parallax direction, the sidesof the respective first members facing to each other, the second memberseach having a side parallel to the orthogonal direction, and the sidesof the respective second members facing to each other.
 13. A diaphragmcontrol method comprising: controlling a distance between centroids of apair of apertures in a diaphragm, the diaphragm forming the pair ofapertures used for generating a stereoscopic picture, wherein thediaphragm includes a first member that includes a first blade and asecond blade each having a cutout and a second member that includes athird blade and a fourth blade each having a projection, the first bladeand the second blade being disposed to allow the cutouts to face eachother, and the third blade and the fourth blade being disposed to allowthe projections to face each other; driving the first member and thesecond member in an orthogonal direction orthogonal to a parallaxdirection; and controlling an increase and decrease of a light amount ofobject light that passes through the pair of apertures, independently ofthe distance between the centroids, wherein the cutout has a concaveshape in which an apex of a mountain shape is located on a straight linethat passes through a midpoint of the distance between the centroids andis parallel to a driving direction of the first member, the projectionhas a convex shape in which an apex of a mountain shape is located on astraight line that passes through the midpoint of the distance betweenthe centroids and is parallel to a driving direction of the secondmember, and the distance between the centroids and the increase anddecrease of the light amount are controlled independently of oneanother.
 14. A non-transitory computer-readable medium storinginstructions that, when executed by a processor of a computer, cause thecomputer to execute operations comprising: controlling a distancebetween centroids of a pair of apertures in a diaphragm, the diaphragmforming the pair of apertures used for generating a stereoscopicpicture, wherein the diaphragm includes a first member that includes afirst blade and a second blade each having a cutout, and a second memberthat includes a third blade and a fourth blade each having a projection,the first blade and the second blade being disposed to allow the cutoutsto face each other, and the third blade and the fourth blade beingdisposed to allow the projections to face each other; driving the firstmember and the second member in an orthogonal direction orthogonal to aparallax direction; and controlling an increase and decrease of a lightamount of object light that passes through the pair of apertures,independently of the distance between the centroids, wherein the cutouthas a concave shape in which an apex of a mountain shape is located on astraight line that passes through a midpoint of the distance between thecentroids and is parallel to a driving direction of the first member,the projection has a convex shape in which an apex of a mountain shapeis located on a straight line that passes through the midpoint of thedistance between the centroids and is parallel to a driving direction ofthe second member, and the distance between the centroids and theincrease and decrease of the light amount are controlled independentlyof one another.